-- Logs begin at Fri 2022-06-24 07:52:48 UTC, end at Fri 2022-06-24 09:01:48 UTC. -- Jun 24 08:26:12.907802 dv-cinder-847131-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 24 08:26:12.915684 dv-cinder-847131-2-iscsi devstack@keystone.service[80413]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 24 08:26:12.917659 dv-cinder-847131-2-iscsi devstack@keystone.service[80413]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Jun 24 08:26:12 2022] *** Jun 24 08:26:12.917659 dv-cinder-847131-2-iscsi devstack@keystone.service[80413]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 24 08:26:12.917659 dv-cinder-847131-2-iscsi devstack@keystone.service[80413]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 24 08:26:12.917659 dv-cinder-847131-2-iscsi devstack@keystone.service[80413]: nodename: dv-cinder-847131-2-iscsi Jun 24 08:26:12.917659 dv-cinder-847131-2-iscsi devstack@keystone.service[80413]: machine: x86_64 Jun 24 08:26:12.917659 dv-cinder-847131-2-iscsi devstack@keystone.service[80413]: clock source: unix Jun 24 08:26:12.917659 dv-cinder-847131-2-iscsi devstack@keystone.service[80413]: pcre jit disabled Jun 24 08:26:12.917659 dv-cinder-847131-2-iscsi devstack@keystone.service[80413]: detected number of CPU cores: 4 Jun 24 08:26:12.918121 dv-cinder-847131-2-iscsi devstack@keystone.service[80413]: current working directory: / Jun 24 08:26:12.918121 dv-cinder-847131-2-iscsi devstack@keystone.service[80413]: detected binary path: /usr/bin/uwsgi-core Jun 24 08:26:12.920032 dv-cinder-847131-2-iscsi devstack@keystone.service[80413]: your processes number limit is 47680 Jun 24 08:26:12.920032 dv-cinder-847131-2-iscsi devstack@keystone.service[80413]: your memory page size is 4096 bytes Jun 24 08:26:12.920032 dv-cinder-847131-2-iscsi devstack@keystone.service[80413]: detected max file descriptor number: 2048 Jun 24 08:26:12.920032 dv-cinder-847131-2-iscsi devstack@keystone.service[80413]: lock engine: pthread robust mutexes Jun 24 08:26:12.920032 dv-cinder-847131-2-iscsi devstack@keystone.service[80413]: thunder lock: enabled Jun 24 08:26:12.920032 dv-cinder-847131-2-iscsi devstack@keystone.service[80413]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 24 08:26:12.920032 dv-cinder-847131-2-iscsi devstack@keystone.service[80413]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 24 08:26:12.962748 dv-cinder-847131-2-iscsi devstack@keystone.service[80413]: Python main interpreter initialized at 0x559f11d849c0 Jun 24 08:26:12.962748 dv-cinder-847131-2-iscsi devstack@keystone.service[80413]: python threads support enabled Jun 24 08:26:12.962748 dv-cinder-847131-2-iscsi devstack@keystone.service[80413]: your server socket listen backlog is limited to 100 connections Jun 24 08:26:12.962748 dv-cinder-847131-2-iscsi devstack@keystone.service[80413]: your mercy for graceful operations on workers is 90 seconds Jun 24 08:26:12.963135 dv-cinder-847131-2-iscsi devstack@keystone.service[80413]: mapped 403077 bytes (393 KB) for 2 cores Jun 24 08:26:12.963209 dv-cinder-847131-2-iscsi devstack@keystone.service[80413]: *** Operational MODE: preforking *** Jun 24 08:26:12.963441 dv-cinder-847131-2-iscsi devstack@keystone.service[80413]: *** uWSGI is running in multiple interpreter mode *** Jun 24 08:26:12.963441 dv-cinder-847131-2-iscsi devstack@keystone.service[80413]: spawned uWSGI master process (pid: 80413) Jun 24 08:26:12.963650 dv-cinder-847131-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 24 08:26:12.969717 dv-cinder-847131-2-iscsi devstack@keystone.service[80413]: spawned uWSGI worker 1 (pid: 80424, cores: 1) Jun 24 08:26:12.969717 dv-cinder-847131-2-iscsi devstack@keystone.service[80413]: spawned uWSGI worker 2 (pid: 80425, cores: 1) Jun 24 08:26:12.969717 dv-cinder-847131-2-iscsi devstack@keystone.service[80413]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 24 08:26:18.476496 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 24 08:26:18.476496 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: from cryptography.utils import int_from_bytes Jun 24 08:26:18.477664 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 24 08:26:18.477664 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: from cryptography.utils import int_from_bytes Jun 24 08:26:18.477884 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 24 08:26:18.477884 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: from cryptography.utils import int_from_bytes Jun 24 08:26:18.477884 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 24 08:26:18.477884 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: from cryptography.utils import int_from_bytes Jun 24 08:26:18.526381 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 24 08:26:18.526572 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 24 08:26:18.526747 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] command line args: [] {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 24 08:26:18.527009 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 24 08:26:18.527125 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 24 08:26:18.527262 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ================================================================================ {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 24 08:26:18.527371 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 24 08:26:18.527496 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] admin_token = **** {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.527613 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] command line args: [] {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 24 08:26:18.527722 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] config_dir = [] {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.527833 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 24 08:26:18.527960 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.528075 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ================================================================================ {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 24 08:26:18.528182 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] config_source = [] {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.528182 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] debug = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.528395 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] admin_token = **** {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.528510 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.528752 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] config_dir = [] {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.528752 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.528945 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.529053 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] config_source = [] {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.529164 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.529272 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] debug = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.529369 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.529462 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.529587 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.529732 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.529732 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.529939 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] list_limit = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.529939 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] log_config_append = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.530130 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.530236 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.530351 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.530455 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] log_dir = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.530455 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] log_file = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.530651 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] list_limit = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.530651 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] log_config_append = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.530827 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.530934 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.531037 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.531133 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] log_dir = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.531226 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.531327 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] log_file = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.531327 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.531518 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.531518 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.531721 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.531839 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.531930 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.532012 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.532102 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.532195 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.532276 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.532350 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.532423 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.532511 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.532591 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.532671 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.532756 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.532835 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.532906 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.533007 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.533088 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.533272 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.533389 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.533497 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.533605 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.533771 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.533984 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.534170 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.534283 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] publish_errors = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.534468 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.534575 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.534575 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.534844 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.534976 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.535108 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] publish_errors = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.535231 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.535394 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.535499 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.535618 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.535772 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] standard_threads = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.535893 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.536011 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.536130 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.536246 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.536444 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.536564 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.536564 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] use_journal = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.536838 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] standard_threads = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.536998 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] use_json = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.537120 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.537294 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] use_stderr = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.537400 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.537551 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] use_syslog = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.537674 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.537877 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.537998 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] use_journal = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.538123 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.538243 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] use_json = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.538243 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] use_stderr = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.538485 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.538590 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] use_syslog = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.538713 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.538830 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:26:18.538945 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.539054 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.539218 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.539866 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.539998 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.540149 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.540257 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.540367 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.540477 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.540586 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.540707 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.540813 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.540972 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.540972 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.541208 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.541322 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.541444 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.541550 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.541659 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.541835 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.541957 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.542064 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.542192 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.542298 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.542416 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.542533 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.542646 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.542758 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.542950 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.543057 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.543192 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.543299 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.543417 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.543539 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.543671 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.543777 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.543890 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.544003 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.544111 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.544221 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.544348 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.544455 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.544574 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.544679 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.544796 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.544911 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.545030 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.545134 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.545244 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.545352 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.545463 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.545573 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.545845 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.545966 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.546083 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.546205 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.546319 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.546442 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.546547 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.546665 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.546770 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.546879 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.546998 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.547115 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.547222 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.547332 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.547440 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.547555 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.547673 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.547790 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.547897 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.548004 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.548113 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.548230 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.548230 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.548509 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.548616 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.548725 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.548834 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.548944 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.549063 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.549187 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.549445 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.549634 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.549754 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.549887 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.549994 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.550115 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.550231 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.550346 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.550452 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.550560 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.550676 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.550800 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.550953 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.551109 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.551262 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.551411 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.551559 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.551717 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.551865 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.552021 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.552175 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.552344 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.552549 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.552675 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.552846 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.552963 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.553117 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.553227 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.553382 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.553514 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.553636 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.553840 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.553947 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.554116 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.554224 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.554417 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.554530 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.555179 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.556094 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.556205 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.556361 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.556474 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.556610 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.556776 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.556895 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.557064 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.557184 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.557352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.557471 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.557646 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.557773 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.557895 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.558059 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.558166 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.558314 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.558438 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.558546 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.558671 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.558779 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.558897 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.559014 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.559149 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.559261 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.559384 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.559504 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.559612 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.559740 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.559847 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.559967 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.560082 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.560082 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.560287 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.560408 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.560515 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.560623 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.560749 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.560858 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.560978 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.561096 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.561205 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.561329 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.561436 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.561587 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.561744 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.561870 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.561977 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.562088 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.562199 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.562309 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.562420 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.562615 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.562733 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.562842 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.562957 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.563076 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.563183 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.563305 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.563444 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.563567 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.563684 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.563792 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.563903 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.564019 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.564128 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.564250 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.564357 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.564468 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.564583 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.564693 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.564804 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.564931 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.565038 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.565148 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.565258 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.565370 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.565480 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.565602 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.565733 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.565855 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.565965 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] auth.external = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.566076 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.566190 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.566316 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.566434 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.566542 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.566652 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.566763 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.566885 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] auth.password = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.567026 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] auth.external = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.567144 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] auth.token = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.567264 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.567373 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.567373 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.567591 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.567708 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.567830 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.567948 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.568056 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] auth.password = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.568167 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.568288 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] auth.token = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.568397 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.568519 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.568628 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.568738 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.568848 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] credential.caching = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.568958 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.569068 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.569192 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.569304 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.569416 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.569526 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.569638 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.569823 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.569945 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.570055 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.570167 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] credential.caching = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.570279 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.570967 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.571495 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.571628 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.571742 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.571865 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.571978 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.572094 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.572209 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.572336 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.572336 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.572555 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.572670 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.572785 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.572900 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.573031 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.573143 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.573260 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.573375 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.573494 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.573494 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] federation.caching = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.573766 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.573881 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.573999 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.574130 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.574243 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.574362 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.574490 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.574605 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.574723 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.574841 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.574959 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] federation.caching = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.575081 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.575211 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.575336 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.575451 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.575569 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.575687 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.575813 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.575939 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.576054 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.576172 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.576290 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.576408 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.576526 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] identity.caching = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.576655 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.576779 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.576897 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.577018 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.577136 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.577253 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.577456 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.577567 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.577684 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.577836 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.577954 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] identity.caching = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.578087 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.578196 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.578313 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.578432 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.578547 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.578663 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.578789 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.578906 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.579024 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.579141 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.579254 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.579254 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.579481 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.579481 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.579698 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.579698 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.579908 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.580026 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.580151 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.580151 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.580363 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.580476 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.580591 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.580706 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.580838 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.580947 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.581062 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.581180 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.581293 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.581408 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.581544 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.581653 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.581830 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.581830 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.582025 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.582137 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.582273 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.582385 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.582504 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.582620 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.582737 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.582853 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.582982 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.583097 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.583219 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.583336 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.583453 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.583572 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.583700 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.583813 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.583934 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.584052 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.584170 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.584288 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.584415 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.584530 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.584656 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.584772 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.584894 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.585012 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.585140 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.585254 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.585372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.585489 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.585612 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.585745 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.585895 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.586005 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.586279 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.586756 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.586891 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.587022 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.587152 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.587266 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.587382 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.587499 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.587617 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.587734 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.587862 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.587975 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.588103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.588215 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.588442 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.588552 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.588687 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.588807 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.588926 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.589044 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.589161 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.589281 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.user = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.589411 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.589537 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.589654 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.589836 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.589955 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.590071 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.590204 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.590313 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.590430 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.590549 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.590664 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.590790 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.590903 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.user = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.591017 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.591132 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.591247 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.591247 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.591475 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.591585 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.591701 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.591816 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.591931 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.592051 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.592178 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.592290 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.592406 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.592520 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.592635 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.592635 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.592861 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.592970 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.592970 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.593185 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.593299 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.593415 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.593541 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.593541 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.593789 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.593905 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.594023 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.594144 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.594273 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.594390 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.594507 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.594626 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.594753 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.594869 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.594997 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.594997 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.595203 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.595317 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.595317 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.595528 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.595655 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.595655 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.595865 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.595982 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.595982 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.596194 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.596321 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.596437 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.596556 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.596674 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.596797 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.596797 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.597020 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.597134 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.597134 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.597346 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.597461 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] resource.caching = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.597461 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.597714 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.597837 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.597837 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.598052 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.598168 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.598168 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.598392 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.598502 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.598619 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.598739 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.598856 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.598974 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.598974 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.599210 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.599326 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] role.caching = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.599446 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.599565 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] role.driver = sql {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.599682 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.599812 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.599929 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.600044 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.600159 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.600277 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.600277 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.600574 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] resource.caching = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.600685 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.600685 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.600898 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.601016 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.601016 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.601238 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.601360 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.601478 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.601601 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.602297 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.602415 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.602552 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.602673 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.602786 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.602902 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.603019 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.603146 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.603258 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.603258 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.603475 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.603589 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] role.caching = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.603704 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.603836 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] role.driver = sql {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.603948 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.604064 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.604181 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.604301 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.604414 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.604540 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.604652 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.604652 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.604868 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.604980 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.605095 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.605224 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.605341 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.605458 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.605573 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.605706 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.605830 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.605968 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.606080 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.606200 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.606320 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.606320 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.606533 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.606659 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.606776 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.606899 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.607019 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.607135 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.607253 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.607253 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.607483 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.607601 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.607719 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.607835 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] token.caching = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.607953 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.608082 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.608197 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.608316 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.608438 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.608557 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.608675 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.608806 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.608922 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.609043 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.609159 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.609276 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.609276 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.609511 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.609630 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.609780 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.609894 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.610011 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.610148 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.610259 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.610374 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.610489 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.610603 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.610718 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.610845 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.610959 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.611074 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.611189 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.611304 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.611419 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.611545 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.611658 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.611778 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.611893 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.611893 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] token.caching = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.612109 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] database.connection = **** {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.612243 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.612364 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.612488 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.612601 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.612716 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.612831 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.612831 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.613047 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.613173 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.613289 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.613405 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.613522 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.613637 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.613773 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.613913 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.614032 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.614152 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.614270 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.614448 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.614558 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.614690 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.614807 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.614927 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.615050 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.615170 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.615289 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.615418 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.615542 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.615660 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.615779 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.615905 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.616022 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.616152 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.616269 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.616387 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.616387 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 24 08:26:18.616600 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] database.connection = **** {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.616600 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.616600 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.616600 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.616600 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.616600 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.616600 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.616600 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.616600 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.618569 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.618569 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.618569 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.618569 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.618569 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.618569 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.618569 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.618569 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.618569 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.619468 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:26:18.619468 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 24 08:26:18.643403 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:18.643589 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:18.643778 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:18.646861 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:18.647099 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:18.647330 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:18.652330 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:18.656340 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:18.681345 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:18.681567 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:18.686553 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:18.686790 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:18.824478 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:18.825390 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:18.879456 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:18.879681 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:18.883294 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:18.883783 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:18.946062 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:18.946493 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:18.946870 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:18.947017 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:18.947311 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:18.947461 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:18.947598 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:18.947735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:18.947873 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:18.948021 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:18.948327 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:18.948627 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:18.973542 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:18.973768 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:18.973927 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:18.974047 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:18.974173 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:18.974297 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:18.974421 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:18.974562 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:18.974679 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:18.974800 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:18.974919 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:18.975040 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:18.975160 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:18.975291 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:18.986692 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:18.987034 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:19.000024 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:19.000260 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:19.000451 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:19.000571 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:19.000696 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:19.000820 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:19.000948 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:19.001080 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:19.001080 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:19.001293 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:19.001410 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:19.001530 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:19.001530 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:19.001870 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:26:19.020309 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.020522 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 24 08:26:19.020860 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.021022 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.021158 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 24 08:26:19.021276 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 24 08:26:19.021399 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.021557 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.021670 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.021822 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 24 08:26:19.021965 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.022079 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.022197 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.022315 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.022547 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.022808 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.033978 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.034255 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.034369 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.034522 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.034636 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.034805 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.034920 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.035152 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.043510 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:26:19.043633 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:26:19.046882 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:26:19.047002 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:26:19.047160 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.047275 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.047439 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.047570 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.047723 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.047838 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.047999 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.048111 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.048244 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.048357 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.048507 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.048620 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.048778 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.048892 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.049047 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.049161 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.049313 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.049423 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.049572 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.049736 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.078428 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.079061 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.080046 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:26:19.080324 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.080681 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:26:19.080959 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.085558 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:26:19.086019 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.086469 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:26:19.086812 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.086951 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.087218 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.094136 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:26:19.094417 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.096186 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:26:19.096472 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.098871 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:26:19.099157 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.099439 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.099720 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.099993 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.100264 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.100530 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.100795 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.100967 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:26:19.101251 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.101531 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.101913 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.102232 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.102538 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.102810 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.103077 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.120984 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.121268 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.122834 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.123110 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.126295 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:26:19.127738 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:26:19.129927 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.130325 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.131386 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.131783 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.136511 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:26:19.137625 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:26:19.140101 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:26:19.141241 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:26:19.143786 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.144081 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.144362 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.144641 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.144907 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.145022 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.145161 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.145277 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.145478 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.145788 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.146081 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.146357 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.165212 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:26:19.165497 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.165611 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:26:19.165794 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.165925 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.166070 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.166183 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.166419 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.173916 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.174439 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.175624 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.175897 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.176145 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.176429 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.179134 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:26:19.179435 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.179648 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:26:19.180435 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.180561 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.180747 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.180970 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.181088 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.181310 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.181423 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.194476 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:26:19.194621 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:26:19.194742 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.194877 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.195028 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.195142 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.195326 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.195467 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.197088 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.197307 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.197424 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.197608 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.197759 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.197934 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.215896 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:26:19.216810 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:26:19.219232 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:26:19.220122 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:26:19.222549 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.223430 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.224292 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.225153 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.225955 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:26:19.226233 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.226512 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.227253 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:26:19.227538 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.227818 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.233873 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.235229 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.235489 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:26:19.236833 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:26:19.238757 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.239055 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.239444 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.239721 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.240078 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.240370 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.240762 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.241034 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.249585 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:26:19.249893 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.250171 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.251420 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:26:19.251707 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.251985 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.258085 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:26:19.258365 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.258819 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.259100 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.259371 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.259763 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.260043 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.260314 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.260487 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:26:19.260598 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.260767 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.260881 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.261049 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.261170 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.261328 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.261453 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.261583 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.261795 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.261908 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.262024 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.262258 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.262537 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.262809 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.263077 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.263358 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.263629 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.263893 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.264159 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:26:19.300132 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x559f11d849c0 pid: 80425 (default app) Jun 24 08:26:19.303436 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x559f11d849c0 pid: 80424 (default app) Jun 24 08:26:19.303603 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6356b0d0-180f-4dc1-8e3b-1e5e331a1c1c None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:19.303907 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6356b0d0-180f-4dc1-8e3b-1e5e331a1c1c None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:19.304189 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6356b0d0-180f-4dc1-8e3b-1e5e331a1c1c None None] PATH_INFO: `/v3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:19.308910 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1/1] 10.222.0.13 () {50 vars in 791 bytes} [Fri Jun 24 08:26:19 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:26:27.177080 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0737343f-588f-4051-86d9-36108bfd2fa8 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:27.177661 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0737343f-588f-4051-86d9-36108bfd2fa8 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:27.177973 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0737343f-588f-4051-86d9-36108bfd2fa8 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:27.185376 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1/2] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:26:27 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:26:27.192347 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393346af-7374-421b-8cc7-6e9940a15430 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:27.192781 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393346af-7374-421b-8cc7-6e9940a15430 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:27.192898 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393346af-7374-421b-8cc7-6e9940a15430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:27.267658 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: INFO dbcounter [None req-393346af-7374-421b-8cc7-6e9940a15430 None None] Registered counter for database keystone Jun 24 08:26:27.290068 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG oslo_db.sqlalchemy.engines [None req-393346af-7374-421b-8cc7-6e9940a15430 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80425) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 24 08:26:27.296322 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writer thread running {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 24 08:26:27.472444 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG passlib.handlers.bcrypt [None req-393346af-7374-421b-8cc7-6e9940a15430 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80425) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 24 08:26:27.472777 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG passlib.handlers.bcrypt [None req-393346af-7374-421b-8cc7-6e9940a15430 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80425) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 24 08:26:27.527424 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-393346af-7374-421b-8cc7-6e9940a15430 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:26:27.529004 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-393346af-7374-421b-8cc7-6e9940a15430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:26:27.650139 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 2/3] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:26:27 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 459 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 24 08:26:27.796516 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0737343f-588f-4051-86d9-36108bfd2fa8 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:27.866558 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: INFO dbcounter [None req-0737343f-588f-4051-86d9-36108bfd2fa8 None None] Registered counter for database keystone Jun 24 08:26:27.886076 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG oslo_db.sqlalchemy.engines [None req-0737343f-588f-4051-86d9-36108bfd2fa8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80424) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 24 08:26:27.892351 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writer thread running {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 24 08:26:28.013373 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0737343f-588f-4051-86d9-36108bfd2fa8 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:28.017712 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31d763b-f936-4b3c-ad2d-26087621f8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:28.018387 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d763b-f936-4b3c-ad2d-26087621f8e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:28.018677 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d763b-f936-4b3c-ad2d-26087621f8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:28.018955 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d763b-f936-4b3c-ad2d-26087621f8e5 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:28.027810 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31d763b-f936-4b3c-ad2d-26087621f8e5 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:28.079685 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f31d763b-f936-4b3c-ad2d-26087621f8e5 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 24 08:26:28.079970 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31d763b-f936-4b3c-ad2d-26087621f8e5 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:28.090628 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 2/4] 10.222.0.13 () {58 vars in 1324 bytes} [Fri Jun 24 08:26:27 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 08:26:28.096474 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393346af-7374-421b-8cc7-6e9940a15430 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:28.111479 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393346af-7374-421b-8cc7-6e9940a15430 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:28.115814 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb96d717-67de-4c73-b1da-88af530d3812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:28.116916 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb96d717-67de-4c73-b1da-88af530d3812 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:28.117293 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb96d717-67de-4c73-b1da-88af530d3812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:28.118149 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb96d717-67de-4c73-b1da-88af530d3812 admin admin] PATH_INFO: `/v3/projects/985b4af5ece540be8554aecad33b1dc4` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:28.119673 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb96d717-67de-4c73-b1da-88af530d3812 admin admin] RBAC: Authorizing `identity:get_project(project_id=985b4af5ece540be8554aecad33b1dc4)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:28.172112 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: WARNING keystone.common.rbac_enforcer.enforcer [None req-eb96d717-67de-4c73-b1da-88af530d3812 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 24 08:26:28.172448 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb96d717-67de-4c73-b1da-88af530d3812 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:28.176963 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 3/5] 10.222.0.13 () {56 vars in 1207 bytes} [Fri Jun 24 08:26:28 2022] GET /identity/v3/projects/985b4af5ece540be8554aecad33b1dc4 => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:26:30.171174 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd95d5-3609-4d7f-9304-8294807722fe None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:30.171710 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd95d5-3609-4d7f-9304-8294807722fe None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:30.171710 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd95d5-3609-4d7f-9304-8294807722fe None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:30.175797 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 3/6] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:26:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:26:30.181721 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4b49d-a0bd-41e8-b8aa-56d184bfd158 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:30.182154 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4b49d-a0bd-41e8-b8aa-56d184bfd158 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:30.182304 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4b49d-a0bd-41e8-b8aa-56d184bfd158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:30.201492 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-51b4b49d-a0bd-41e8-b8aa-56d184bfd158 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:26:30.203046 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-51b4b49d-a0bd-41e8-b8aa-56d184bfd158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:26:30.210938 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 4/7] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:26:30 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 24 08:26:30.232903 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cd95d5-3609-4d7f-9304-8294807722fe None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:30.245299 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cd95d5-3609-4d7f-9304-8294807722fe None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:30.249717 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b738078-382b-4feb-b5a8-62daf80e0ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:30.250349 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b738078-382b-4feb-b5a8-62daf80e0ab3 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:30.250631 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b738078-382b-4feb-b5a8-62daf80e0ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:30.250903 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b738078-382b-4feb-b5a8-62daf80e0ab3 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:30.258567 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b738078-382b-4feb-b5a8-62daf80e0ab3 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:30.260316 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b738078-382b-4feb-b5a8-62daf80e0ab3 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:30.269481 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 4/8] 10.222.0.13 () {58 vars in 1324 bytes} [Fri Jun 24 08:26:30 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 08:26:30.275980 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b4b49d-a0bd-41e8-b8aa-56d184bfd158 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:30.285162 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b4b49d-a0bd-41e8-b8aa-56d184bfd158 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:30.290235 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e15c67d-2f21-40be-8a0b-a0b660b34345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:30.291166 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e15c67d-2f21-40be-8a0b-a0b660b34345 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:30.291449 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e15c67d-2f21-40be-8a0b-a0b660b34345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:30.291722 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e15c67d-2f21-40be-8a0b-a0b660b34345 admin admin] PATH_INFO: `/v3/users/268e75b173ff435b8dfbe2517f69c015` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:30.293117 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e15c67d-2f21-40be-8a0b-a0b660b34345 admin admin] RBAC: Authorizing `identity:get_user(user_id=268e75b173ff435b8dfbe2517f69c015)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:30.294969 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e15c67d-2f21-40be-8a0b-a0b660b34345 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:30.296532 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 5/9] 10.222.0.13 () {56 vars in 1198 bytes} [Fri Jun 24 08:26:30 2022] GET /identity/v3/users/268e75b173ff435b8dfbe2517f69c015 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:26:32.368528 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87146b5c-53b4-4244-9fb1-6465e2010c7b None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:32.369064 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87146b5c-53b4-4244-9fb1-6465e2010c7b None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:32.369064 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87146b5c-53b4-4244-9fb1-6465e2010c7b None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:32.370450 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 5/10] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:26:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:26:32.376043 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f7db7-1046-417c-bcfd-2f3a88e41af3 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:32.376413 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f7db7-1046-417c-bcfd-2f3a88e41af3 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:32.376605 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f7db7-1046-417c-bcfd-2f3a88e41af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:32.393631 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-fa6f7db7-1046-417c-bcfd-2f3a88e41af3 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:26:32.394833 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-fa6f7db7-1046-417c-bcfd-2f3a88e41af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:26:32.442379 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e882e4e-6d21-42b7-92ee-a20e4849a46f None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:32.442661 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e882e4e-6d21-42b7-92ee-a20e4849a46f None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:32.442940 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e882e4e-6d21-42b7-92ee-a20e4849a46f None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:32.444324 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 6/11] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:26:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:26:32.450261 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f106150e-fcdf-4fec-af74-aef7e1a938e2 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:32.450547 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f106150e-fcdf-4fec-af74-aef7e1a938e2 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:32.450829 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f106150e-fcdf-4fec-af74-aef7e1a938e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:32.474732 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 6/12] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:26:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:26:32.490223 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG passlib.handlers.bcrypt [None req-f106150e-fcdf-4fec-af74-aef7e1a938e2 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80424) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 24 08:26:32.490654 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG passlib.handlers.bcrypt [None req-f106150e-fcdf-4fec-af74-aef7e1a938e2 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80424) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 24 08:26:32.498202 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f7db7-1046-417c-bcfd-2f3a88e41af3 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:32.521563 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:32.537295 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-f106150e-fcdf-4fec-af74-aef7e1a938e2 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:26:32.538928 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-f106150e-fcdf-4fec-af74-aef7e1a938e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:26:32.606964 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f7db7-1046-417c-bcfd-2f3a88e41af3 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:32.645737 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 7/13] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:26:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 08:26:32.656174 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24daec4-b7d9-41e7-8fee-08067aa1949e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:32.657463 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24daec4-b7d9-41e7-8fee-08067aa1949e None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:32.657850 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24daec4-b7d9-41e7-8fee-08067aa1949e None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:32.658190 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24daec4-b7d9-41e7-8fee-08067aa1949e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:32.669322 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f106150e-fcdf-4fec-af74-aef7e1a938e2 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:32.672955 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24daec4-b7d9-41e7-8fee-08067aa1949e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:26:32.684660 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-d24daec4-b7d9-41e7-8fee-08067aa1949e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:26:32.684660 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:26:32.684660 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:26:32.684660 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:26:32.684660 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:26:32.684660 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:26:32.684660 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:26:32.684660 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:26:32.684660 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:26:32.684660 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:26:32.684660 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:26:32.684660 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:26:32.684660 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:26:32.684660 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:26:32.684660 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:26:32.684660 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:26:32.685736 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:26:32.685736 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:26:32.685736 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:26:32.685736 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:26:32.685736 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:26:32.685736 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:26:32.685736 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:26:32.685736 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:26:32.685736 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:26:32.685736 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:26:32.685736 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:26:32.685736 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:26:32.685736 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:26:32.685736 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:26:32.685736 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:26:32.685736 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:26:32.686550 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:26:32.686550 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:26:32.686550 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:26:32.686550 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:26:32.686550 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:26:32.686550 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:26:32.686550 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:26:32.686550 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:26:32.686550 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:26:32.686550 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:26:32.686550 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:26:32.686550 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:26:32.686550 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 7/14] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Jun 24 08:26:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:26:32.691516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24daec4-b7d9-41e7-8fee-08067aa1949e None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:32.770649 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f106150e-fcdf-4fec-af74-aef7e1a938e2 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:32.773244 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:32.774150 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:32.795093 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24daec4-b7d9-41e7-8fee-08067aa1949e None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:32.811742 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e37d983-b943-4ef5-b25e-ff56517d99fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:32.812650 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e37d983-b943-4ef5-b25e-ff56517d99fe None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:32.812943 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e37d983-b943-4ef5-b25e-ff56517d99fe None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:32.813225 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e37d983-b943-4ef5-b25e-ff56517d99fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:32.829803 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a135de2a-90ff-4f4a-96ba-9bc3d312d732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:32.830463 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a135de2a-90ff-4f4a-96ba-9bc3d312d732 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:32.830746 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a135de2a-90ff-4f4a-96ba-9bc3d312d732 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:32.831023 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a135de2a-90ff-4f4a-96ba-9bc3d312d732 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:32.831774 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a135de2a-90ff-4f4a-96ba-9bc3d312d732 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:26:32.832076 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a135de2a-90ff-4f4a-96ba-9bc3d312d732 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:32.833474 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a135de2a-90ff-4f4a-96ba-9bc3d312d732 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:32.845194 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 8/15] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:26:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:26:32.850575 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a135de2a-90ff-4f4a-96ba-9bc3d312d732 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:32.882294 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e37d983-b943-4ef5-b25e-ff56517d99fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:32.884111 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e37d983-b943-4ef5-b25e-ff56517d99fe None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:32.954709 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a135de2a-90ff-4f4a-96ba-9bc3d312d732 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:32.988289 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 8/16] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:26:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 08:26:32.995704 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ea43b-dffb-4138-bae6-134377b96a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:32.996299 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e37d983-b943-4ef5-b25e-ff56517d99fe None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:32.996605 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ea43b-dffb-4138-bae6-134377b96a54 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:32.996867 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ea43b-dffb-4138-bae6-134377b96a54 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:32.997146 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ea43b-dffb-4138-bae6-134377b96a54 None admin] PATH_INFO: `/v3/users/268e75b173ff435b8dfbe2517f69c015` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:32.999280 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0ea43b-dffb-4138-bae6-134377b96a54 None admin] RBAC: Authorizing `identity:get_user(user_id=268e75b173ff435b8dfbe2517f69c015)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:32.999476 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:33.000895 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0ea43b-dffb-4138-bae6-134377b96a54 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:33.002554 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 9/17] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Jun 24 08:26:32 2022] GET /identity/v3/users/268e75b173ff435b8dfbe2517f69c015 => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:26:33.008388 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ea43b-dffb-4138-bae6-134377b96a54 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:33.024822 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:33.098253 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e37d983-b943-4ef5-b25e-ff56517d99fe None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:33.109232 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ea43b-dffb-4138-bae6-134377b96a54 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:33.132547 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f92acf-d4b6-4c3d-8b7d-750a67eeb3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:33.133455 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f92acf-d4b6-4c3d-8b7d-750a67eeb3d6 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:33.133769 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f92acf-d4b6-4c3d-8b7d-750a67eeb3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:33.134053 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f92acf-d4b6-4c3d-8b7d-750a67eeb3d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:33.141632 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad92ad5-53be-411a-8967-0529a418a0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:33.142439 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad92ad5-53be-411a-8967-0529a418a0a7 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:33.142723 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad92ad5-53be-411a-8967-0529a418a0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:33.142996 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad92ad5-53be-411a-8967-0529a418a0a7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:33.144201 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad92ad5-53be-411a-8967-0529a418a0a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:33.145652 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad92ad5-53be-411a-8967-0529a418a0a7 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:33.147489 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 10/18] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:26:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:26:33.153431 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad92ad5-53be-411a-8967-0529a418a0a7 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:33.154674 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f92acf-d4b6-4c3d-8b7d-750a67eeb3d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:33.156158 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f92acf-d4b6-4c3d-8b7d-750a67eeb3d6 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:33.178192 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-f9f92acf-d4b6-4c3d-8b7d-750a67eeb3d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:26:33.178192 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:26:33.178192 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:26:33.178192 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:26:33.178192 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:26:33.178192 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:26:33.178192 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:26:33.178192 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:26:33.178192 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:26:33.178192 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:26:33.178192 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:26:33.178192 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:26:33.178192 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:26:33.178192 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:26:33.178192 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:26:33.179048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:26:33.179048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:26:33.179048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:26:33.179048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:26:33.179048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:26:33.179048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:26:33.179048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:26:33.179048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:26:33.179048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:26:33.179048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:26:33.179048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:26:33.179048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:26:33.179048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:26:33.179048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:26:33.179048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:26:33.179048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:26:33.179048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:26:33.179906 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:26:33.179906 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:26:33.179906 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:26:33.179906 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:26:33.179906 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 9/19] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:26:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:26:33.185371 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f92acf-d4b6-4c3d-8b7d-750a67eeb3d6 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:33.238049 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:33.243926 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:33.258630 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad92ad5-53be-411a-8967-0529a418a0a7 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:33.294334 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d17c679-ed37-4072-b3d4-87a71242a9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:33.294989 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d17c679-ed37-4072-b3d4-87a71242a9fb None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:33.295278 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d17c679-ed37-4072-b3d4-87a71242a9fb None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:33.295554 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d17c679-ed37-4072-b3d4-87a71242a9fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:33.296377 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d17c679-ed37-4072-b3d4-87a71242a9fb None admin] RBAC: Adding query filter params (role.id=acfe707eee6944ad94d1915fc9ee7ea3, scope.domain.id=default, user.id=268e75b173ff435b8dfbe2517f69c015) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:26:33.296687 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d17c679-ed37-4072-b3d4-87a71242a9fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:33.297330 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f92acf-d4b6-4c3d-8b7d-750a67eeb3d6 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:33.298162 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d17c679-ed37-4072-b3d4-87a71242a9fb None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:33.317725 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 11/20] 10.222.0.13 () {56 vars in 1322 bytes} [Fri Jun 24 08:26:33 2022] GET /identity/v3/role_assignments?user.id=268e75b173ff435b8dfbe2517f69c015&scope.domain.id=default&role.id=acfe707eee6944ad94d1915fc9ee7ea3 => generated 233 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:26:33.332863 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f300f5db-b786-4405-9360-aed2096a6262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:33.333556 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f300f5db-b786-4405-9360-aed2096a6262 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:33.333888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f300f5db-b786-4405-9360-aed2096a6262 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:33.334186 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f300f5db-b786-4405-9360-aed2096a6262 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:33.334932 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f300f5db-b786-4405-9360-aed2096a6262 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:26:33.335246 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f300f5db-b786-4405-9360-aed2096a6262 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:33.337447 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f300f5db-b786-4405-9360-aed2096a6262 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:33.357542 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 10/21] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:26:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:26:35.699624 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b648b-e860-466c-9044-ea3d75ebe061 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:35.700638 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b648b-e860-466c-9044-ea3d75ebe061 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:35.701221 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b648b-e860-466c-9044-ea3d75ebe061 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:35.702811 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 12/22] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:26:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:26:35.708266 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee45da-103d-421d-8d18-5767014efb53 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:35.709085 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee45da-103d-421d-8d18-5767014efb53 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:35.709555 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee45da-103d-421d-8d18-5767014efb53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:35.730720 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-a4ee45da-103d-421d-8d18-5767014efb53 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:26:35.732254 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-a4ee45da-103d-421d-8d18-5767014efb53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:26:35.819123 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 11/23] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:26:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:26:35.840791 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad2355-2ce9-4c23-aa3d-88c08cc03967 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:35.841362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad2355-2ce9-4c23-aa3d-88c08cc03967 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:35.841877 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad2355-2ce9-4c23-aa3d-88c08cc03967 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:35.843453 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 13/24] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:26:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:26:35.848544 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c61c40-3983-4250-b1ea-980567e6f7ba None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:35.849064 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c61c40-3983-4250-b1ea-980567e6f7ba None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:35.849552 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c61c40-3983-4250-b1ea-980567e6f7ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:35.852720 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dad2355-2ce9-4c23-aa3d-88c08cc03967 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:35.896029 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-42c61c40-3983-4250-b1ea-980567e6f7ba None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:26:35.898532 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-42c61c40-3983-4250-b1ea-980567e6f7ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:26:35.986759 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:36.048076 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 12/25] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:26:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:26:36.049394 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b1ad63-7dce-4dc8-8782-f038cb6a2bb6 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:36.049684 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b1ad63-7dce-4dc8-8782-f038cb6a2bb6 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:36.049992 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b1ad63-7dce-4dc8-8782-f038cb6a2bb6 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:36.053340 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 13/26] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:26:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:26:36.061715 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613fdb15-5eec-464f-ac87-0d2018af3d01 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:36.062004 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613fdb15-5eec-464f-ac87-0d2018af3d01 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:36.062283 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613fdb15-5eec-464f-ac87-0d2018af3d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:36.068884 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dad2355-2ce9-4c23-aa3d-88c08cc03967 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:36.089790 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-613fdb15-5eec-464f-ac87-0d2018af3d01 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:26:36.091184 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-613fdb15-5eec-464f-ac87-0d2018af3d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:26:36.122065 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2ad4b0-ed2c-4ae9-a6ec-608ed591736c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:36.123150 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ad4b0-ed2c-4ae9-a6ec-608ed591736c None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:36.123722 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ad4b0-ed2c-4ae9-a6ec-608ed591736c None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:36.124005 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ad4b0-ed2c-4ae9-a6ec-608ed591736c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:36.151680 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2ad4b0-ed2c-4ae9-a6ec-608ed591736c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:26:36.159341 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-0e2ad4b0-ed2c-4ae9-a6ec-608ed591736c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:26:36.159341 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:26:36.159341 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:26:36.159341 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:26:36.159341 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:26:36.159341 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:26:36.159341 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:26:36.159341 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:26:36.159341 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:26:36.159341 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:26:36.159341 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:26:36.159341 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:26:36.159341 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:26:36.159341 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:26:36.159341 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:26:36.159341 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:26:36.160182 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:26:36.160182 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:26:36.160182 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:26:36.160182 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:26:36.160182 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:26:36.160182 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:26:36.160182 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:26:36.160182 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:26:36.160182 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:26:36.160182 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:26:36.160182 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:26:36.160182 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:26:36.160182 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:26:36.160182 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:26:36.160182 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:26:36.160182 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:26:36.160934 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:26:36.160934 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:26:36.160934 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:26:36.160934 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:26:36.160934 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:26:36.160934 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:26:36.160934 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:26:36.160934 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:26:36.160934 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:26:36.160934 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:26:36.160934 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:26:36.160934 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:26:36.162341 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 14/27] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Jun 24 08:26:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:26:36.168077 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2ad4b0-ed2c-4ae9-a6ec-608ed591736c None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:36.182382 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:36.250410 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 14/28] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:26:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:26:36.251284 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613fdb15-5eec-464f-ac87-0d2018af3d01 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:36.290731 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2ad4b0-ed2c-4ae9-a6ec-608ed591736c None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:36.331300 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58116f7e-587c-4c1d-b7c6-4d44fb5ba1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:36.332334 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58116f7e-587c-4c1d-b7c6-4d44fb5ba1ec None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:36.333509 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58116f7e-587c-4c1d-b7c6-4d44fb5ba1ec None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:36.333966 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58116f7e-587c-4c1d-b7c6-4d44fb5ba1ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:36.355363 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58116f7e-587c-4c1d-b7c6-4d44fb5ba1ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:36.357308 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58116f7e-587c-4c1d-b7c6-4d44fb5ba1ec None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:36.385746 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-58116f7e-587c-4c1d-b7c6-4d44fb5ba1ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:26:36.385746 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:26:36.385746 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:26:36.385746 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:26:36.385746 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:26:36.385746 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:26:36.385746 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:26:36.385746 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:26:36.385746 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:26:36.385746 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:26:36.385746 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:26:36.385746 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:26:36.385746 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:26:36.385746 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:26:36.385746 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:26:36.386752 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:26:36.386752 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:26:36.386752 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:26:36.386752 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:26:36.386752 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:26:36.386752 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:26:36.386752 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:26:36.386752 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:26:36.386752 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:26:36.386752 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:26:36.386752 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:26:36.386752 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:26:36.386752 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:26:36.386752 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:26:36.386752 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:26:36.386752 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:26:36.386752 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:26:36.387793 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:26:36.387793 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:26:36.387793 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:26:36.387793 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:26:36.388628 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 15/29] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:26:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:26:36.389244 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58116f7e-587c-4c1d-b7c6-4d44fb5ba1ec None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:36.457187 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:36.509434 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58116f7e-587c-4c1d-b7c6-4d44fb5ba1ec None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:36.535145 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613fdb15-5eec-464f-ac87-0d2018af3d01 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:36.547134 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda2572a-d81a-4884-ba94-0bc8929f0076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:36.547820 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2572a-d81a-4884-ba94-0bc8929f0076 None admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:36.548106 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2572a-d81a-4884-ba94-0bc8929f0076 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:36.548384 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2572a-d81a-4884-ba94-0bc8929f0076 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:36.549137 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda2572a-d81a-4884-ba94-0bc8929f0076 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:36.550563 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda2572a-d81a-4884-ba94-0bc8929f0076 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:36.551820 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:36.630421 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2aee51-12d6-4e2f-a8a6-af251f50543a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:36.631519 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2aee51-12d6-4e2f-a8a6-af251f50543a None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:36.632033 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2aee51-12d6-4e2f-a8a6-af251f50543a None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:36.632509 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2aee51-12d6-4e2f-a8a6-af251f50543a None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:36.633513 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2aee51-12d6-4e2f-a8a6-af251f50543a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:26:36.634041 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2aee51-12d6-4e2f-a8a6-af251f50543a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:36.635657 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2aee51-12d6-4e2f-a8a6-af251f50543a None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:36.662894 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 15/30] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:26:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 412 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:26:36.664687 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2aee51-12d6-4e2f-a8a6-af251f50543a None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:36.690442 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 16/31] 10.222.0.13 () {60 vars in 1131 bytes} [Fri Jun 24 08:26:36 2022] POST /identity/v3/roles => generated 219 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:26:36.691504 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda2572a-d81a-4884-ba94-0bc8929f0076 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:36.852998 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda2572a-d81a-4884-ba94-0bc8929f0076 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:36.887772 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2aee51-12d6-4e2f-a8a6-af251f50543a None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:36.892480 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46890ec3-88b6-46bb-801a-08a30d8249a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:36.893821 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46890ec3-88b6-46bb-801a-08a30d8249a1 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:36.894286 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46890ec3-88b6-46bb-801a-08a30d8249a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:36.894719 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46890ec3-88b6-46bb-801a-08a30d8249a1 None admin] PATH_INFO: `/v3/users/268e75b173ff435b8dfbe2517f69c015` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:36.896293 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46890ec3-88b6-46bb-801a-08a30d8249a1 None admin] RBAC: Authorizing `identity:get_user(user_id=268e75b173ff435b8dfbe2517f69c015)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:36.898037 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46890ec3-88b6-46bb-801a-08a30d8249a1 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:36.900820 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 17/32] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Jun 24 08:26:36 2022] GET /identity/v3/users/268e75b173ff435b8dfbe2517f69c015 => generated 247 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:26:36.901197 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:36.905936 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d9affd-5064-446a-a705-15baad990a92 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:36.906394 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d9affd-5064-446a-a705-15baad990a92 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:36.906814 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d9affd-5064-446a-a705-15baad990a92 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:36.908590 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 18/33] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:26:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:26:36.909468 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d9affd-5064-446a-a705-15baad990a92 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:36.940518 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb82e6b2-0e28-4237-9066-daf5fe93a4e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:36.941237 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb82e6b2-0e28-4237-9066-daf5fe93a4e8 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:36.941526 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb82e6b2-0e28-4237-9066-daf5fe93a4e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:36.941829 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb82e6b2-0e28-4237-9066-daf5fe93a4e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:36.942753 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb82e6b2-0e28-4237-9066-daf5fe93a4e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:26:36.943063 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb82e6b2-0e28-4237-9066-daf5fe93a4e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:36.944702 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb82e6b2-0e28-4237-9066-daf5fe93a4e8 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:36.957186 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 16/34] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:26:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:26:36.958671 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5998d74-52a1-49c0-b01f-cd8eba17221d None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:36.958966 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5998d74-52a1-49c0-b01f-cd8eba17221d None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:36.959290 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5998d74-52a1-49c0-b01f-cd8eba17221d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:36.977553 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-e5998d74-52a1-49c0-b01f-cd8eba17221d None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:26:36.978709 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-e5998d74-52a1-49c0-b01f-cd8eba17221d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:26:37.011219 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:37.048103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d9affd-5064-446a-a705-15baad990a92 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:37.074607 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 17/35] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:26:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:26:37.075261 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5998d74-52a1-49c0-b01f-cd8eba17221d None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:37.096587 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e7a758-b69c-42a5-887d-4b3ae2a33fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:37.102441 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7a758-b69c-42a5-887d-4b3ae2a33fbd None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:37.102607 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7a758-b69c-42a5-887d-4b3ae2a33fbd None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:37.102892 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7a758-b69c-42a5-887d-4b3ae2a33fbd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:37.106467 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e7a758-b69c-42a5-887d-4b3ae2a33fbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:37.109045 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e7a758-b69c-42a5-887d-4b3ae2a33fbd None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:37.118738 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 19/36] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:26:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:26:37.119281 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e7a758-b69c-42a5-887d-4b3ae2a33fbd None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:37.183802 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:37.210185 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5998d74-52a1-49c0-b01f-cd8eba17221d None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:37.247317 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e7a758-b69c-42a5-887d-4b3ae2a33fbd None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:37.249655 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cade568-d65f-4fba-a853-557ba2fd9934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:37.250632 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cade568-d65f-4fba-a853-557ba2fd9934 None admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:37.251108 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cade568-d65f-4fba-a853-557ba2fd9934 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:37.251594 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cade568-d65f-4fba-a853-557ba2fd9934 None admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:37.252753 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cade568-d65f-4fba-a853-557ba2fd9934 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:37.254482 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cade568-d65f-4fba-a853-557ba2fd9934 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:37.287244 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32a991c-6011-4b93-9bcd-dce9891f1590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:37.288124 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32a991c-6011-4b93-9bcd-dce9891f1590 None admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:37.288412 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32a991c-6011-4b93-9bcd-dce9891f1590 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:37.288692 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32a991c-6011-4b93-9bcd-dce9891f1590 None admin] PATH_INFO: `/v3/domains/default/users/268e75b173ff435b8dfbe2517f69c015/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:37.291244 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32a991c-6011-4b93-9bcd-dce9891f1590 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=268e75b173ff435b8dfbe2517f69c015, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:37.292725 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32a991c-6011-4b93-9bcd-dce9891f1590 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:37.307428 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 20/37] 10.222.0.13 () {58 vars in 1361 bytes} [Fri Jun 24 08:26:37 2022] PUT /identity/v3/domains/default/users/268e75b173ff435b8dfbe2517f69c015/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:26:37.308216 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32a991c-6011-4b93-9bcd-dce9891f1590 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:37.405740 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 18/38] 10.222.0.13 () {60 vars in 1141 bytes} [Fri Jun 24 08:26:37 2022] POST /identity/v3/projects => generated 301 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:26:37.405740 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41107c3-d9a8-4aa4-a3ef-67eaf855e0d4 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:37.405740 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41107c3-d9a8-4aa4-a3ef-67eaf855e0d4 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:37.405740 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41107c3-d9a8-4aa4-a3ef-67eaf855e0d4 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:37.405740 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 19/39] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:26:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:26:37.405740 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e95d44b-d19a-4e06-87b3-aa0de3f89efa None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:37.405740 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e95d44b-d19a-4e06-87b3-aa0de3f89efa None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:37.406372 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e95d44b-d19a-4e06-87b3-aa0de3f89efa None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:37.406372 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 20/40] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:26:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:26:37.412426 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aac1b0-2267-4dfb-99bd-bfe0d9e25629 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:37.412793 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aac1b0-2267-4dfb-99bd-bfe0d9e25629 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:37.412997 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aac1b0-2267-4dfb-99bd-bfe0d9e25629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:37.434166 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-61aac1b0-2267-4dfb-99bd-bfe0d9e25629 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:26:37.435837 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-61aac1b0-2267-4dfb-99bd-bfe0d9e25629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:26:37.439796 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:37.528168 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 21/41] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:26:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 08:26:37.529287 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e5f19-cdf7-4926-bb25-5943910267ab None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:37.529582 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e5f19-cdf7-4926-bb25-5943910267ab None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:37.529894 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e5f19-cdf7-4926-bb25-5943910267ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:37.548743 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-d22e5f19-cdf7-4926-bb25-5943910267ab None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:26:37.553749 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-d22e5f19-cdf7-4926-bb25-5943910267ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:26:37.574109 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32a991c-6011-4b93-9bcd-dce9891f1590 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:37.589596 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:37.620656 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a6fb3c-1996-4451-a721-677a05ac76c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:37.622430 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a6fb3c-1996-4451-a721-677a05ac76c9 None admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:37.625894 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a6fb3c-1996-4451-a721-677a05ac76c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:37.626178 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a6fb3c-1996-4451-a721-677a05ac76c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:37.626940 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a6fb3c-1996-4451-a721-677a05ac76c9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:37.628355 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a6fb3c-1996-4451-a721-677a05ac76c9 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:37.643799 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 21/42] 10.222.0.13 () {60 vars in 1131 bytes} [Fri Jun 24 08:26:37 2022] POST /identity/v3/roles => generated 225 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:26:37.644664 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a6fb3c-1996-4451-a721-677a05ac76c9 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:37.645109 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 22/43] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:26:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:26:37.669867 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e5f19-cdf7-4926-bb25-5943910267ab None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:37.778404 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:37.799198 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a6fb3c-1996-4451-a721-677a05ac76c9 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:37.824968 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e5f19-cdf7-4926-bb25-5943910267ab None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:37.850117 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c2bea8-0485-409b-a0b4-26e4319a92ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:37.851146 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2bea8-0485-409b-a0b4-26e4319a92ad None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:37.851571 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2bea8-0485-409b-a0b4-26e4319a92ad None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:37.852271 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2bea8-0485-409b-a0b4-26e4319a92ad None admin] PATH_INFO: `/v3/domains/default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:37.853829 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c2bea8-0485-409b-a0b4-26e4319a92ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:37.855948 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c2bea8-0485-409b-a0b4-26e4319a92ad None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:37.857842 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 22/44] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:26:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:26:37.863323 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c2bea8-0485-409b-a0b4-26e4319a92ad None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:37.885860 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3da5b8-9724-44e8-a332-f710e8bf1f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:37.886751 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3da5b8-9724-44e8-a332-f710e8bf1f24 None admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:37.887056 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3da5b8-9724-44e8-a332-f710e8bf1f24 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:37.887343 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3da5b8-9724-44e8-a332-f710e8bf1f24 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:37.888616 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3da5b8-9724-44e8-a332-f710e8bf1f24 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:37.890268 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3da5b8-9724-44e8-a332-f710e8bf1f24 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:37.907119 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 23/45] 10.222.0.13 () {60 vars in 1131 bytes} [Fri Jun 24 08:26:37 2022] POST /identity/v3/roles => generated 223 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:26:37.915920 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:37.984484 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c2bea8-0485-409b-a0b4-26e4319a92ad None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:38.025246 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5e02d5-8b76-4b1b-8cc3-a6187d423b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:38.025963 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e02d5-8b76-4b1b-8cc3-a6187d423b7b None admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:38.026257 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e02d5-8b76-4b1b-8cc3-a6187d423b7b None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:38.026540 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e02d5-8b76-4b1b-8cc3-a6187d423b7b None admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:38.027274 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5e02d5-8b76-4b1b-8cc3-a6187d423b7b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:38.028773 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5e02d5-8b76-4b1b-8cc3-a6187d423b7b None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:38.064548 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 23/46] 10.222.0.13 () {60 vars in 1141 bytes} [Fri Jun 24 08:26:37 2022] POST /identity/v3/projects => generated 312 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:26:39.440995 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acaf460-221d-4176-93e1-615ac71f3373 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:39.441495 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acaf460-221d-4176-93e1-615ac71f3373 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:39.441573 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acaf460-221d-4176-93e1-615ac71f3373 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:39.442978 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 24/47] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:26:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:26:39.449320 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007aed2-40bb-40ec-acde-49f1b8a04d9e None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:39.452240 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007aed2-40bb-40ec-acde-49f1b8a04d9e None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:39.452240 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007aed2-40bb-40ec-acde-49f1b8a04d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:39.467379 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-c007aed2-40bb-40ec-acde-49f1b8a04d9e None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:26:39.468897 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-c007aed2-40bb-40ec-acde-49f1b8a04d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:26:39.534791 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 24/48] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:26:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:26:39.558169 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acaf460-221d-4176-93e1-615ac71f3373 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:39.658867 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acaf460-221d-4176-93e1-615ac71f3373 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:39.686273 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:39.692616 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc749a-19e9-485e-89f3-b2b7b5b075cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:39.693445 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc749a-19e9-485e-89f3-b2b7b5b075cb None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:39.693876 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc749a-19e9-485e-89f3-b2b7b5b075cb None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:39.694164 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc749a-19e9-485e-89f3-b2b7b5b075cb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:39.707237 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dc749a-19e9-485e-89f3-b2b7b5b075cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:26:39.720805 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-a1dc749a-19e9-485e-89f3-b2b7b5b075cb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:26:39.720805 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:26:39.720805 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:26:39.720805 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:26:39.720805 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:26:39.720805 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:26:39.720805 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:26:39.720805 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:26:39.720805 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:26:39.720805 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:26:39.720805 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:26:39.720805 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:26:39.720805 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:26:39.720805 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:26:39.720805 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:26:39.720805 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:26:39.721675 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:26:39.721675 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:26:39.721675 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:26:39.721675 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:26:39.721675 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:26:39.721675 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:26:39.721675 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:26:39.721675 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:26:39.721675 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:26:39.721675 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:26:39.721675 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:26:39.721675 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:26:39.721675 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:26:39.721675 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:26:39.721675 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:26:39.721675 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:26:39.722469 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:26:39.722469 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:26:39.722469 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:26:39.722469 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:26:39.722469 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:26:39.722469 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:26:39.722469 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:26:39.722469 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:26:39.722469 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:26:39.722469 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:26:39.722469 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:26:39.722469 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:26:39.722469 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 25/49] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Jun 24 08:26:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:26:39.728701 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007aed2-40bb-40ec-acde-49f1b8a04d9e None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:39.756516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:39.856927 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007aed2-40bb-40ec-acde-49f1b8a04d9e None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:39.893363 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a2e2bd-b01c-4be3-90c4-bc93b10ebf76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:39.894064 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2e2bd-b01c-4be3-90c4-bc93b10ebf76 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:39.894356 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2e2bd-b01c-4be3-90c4-bc93b10ebf76 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:39.894637 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2e2bd-b01c-4be3-90c4-bc93b10ebf76 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:39.895385 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a2e2bd-b01c-4be3-90c4-bc93b10ebf76 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:26:39.895697 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a2e2bd-b01c-4be3-90c4-bc93b10ebf76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:39.897101 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a2e2bd-b01c-4be3-90c4-bc93b10ebf76 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:39.905759 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 25/50] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:26:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:26:39.911092 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc749a-19e9-485e-89f3-b2b7b5b075cb None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:40.017883 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc749a-19e9-485e-89f3-b2b7b5b075cb None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:40.056489 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db92fcb6-54e8-4122-91cf-94e064a47ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:40.057367 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92fcb6-54e8-4122-91cf-94e064a47ebb None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:40.057658 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92fcb6-54e8-4122-91cf-94e064a47ebb None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:40.057967 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92fcb6-54e8-4122-91cf-94e064a47ebb None admin] PATH_INFO: `/v3/users/268e75b173ff435b8dfbe2517f69c015` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:40.059421 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db92fcb6-54e8-4122-91cf-94e064a47ebb None admin] RBAC: Authorizing `identity:get_user(user_id=268e75b173ff435b8dfbe2517f69c015)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:40.060891 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db92fcb6-54e8-4122-91cf-94e064a47ebb None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:40.062573 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 26/51] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Jun 24 08:26:39 2022] GET /identity/v3/users/268e75b173ff435b8dfbe2517f69c015 => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:26:40.067912 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a2e2bd-b01c-4be3-90c4-bc93b10ebf76 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:40.158104 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:40.174951 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a2e2bd-b01c-4be3-90c4-bc93b10ebf76 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:40.185664 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc603a-5baa-405a-b015-1a372cf49d34 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:40.186049 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc603a-5baa-405a-b015-1a372cf49d34 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:40.186380 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc603a-5baa-405a-b015-1a372cf49d34 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:40.188003 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 27/52] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:26:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:26:40.192949 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058eb85-6576-4610-bef6-8b4c60a3cf32 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:40.193247 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058eb85-6576-4610-bef6-8b4c60a3cf32 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:40.193533 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058eb85-6576-4610-bef6-8b4c60a3cf32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:40.208956 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e518b7c-885e-411f-a595-061c5ed5ec83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:40.209772 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e518b7c-885e-411f-a595-061c5ed5ec83 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:40.210057 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e518b7c-885e-411f-a595-061c5ed5ec83 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:40.210332 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e518b7c-885e-411f-a595-061c5ed5ec83 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:40.210797 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-a058eb85-6576-4610-bef6-8b4c60a3cf32 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:26:40.212025 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e518b7c-885e-411f-a595-061c5ed5ec83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:40.212522 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-a058eb85-6576-4610-bef6-8b4c60a3cf32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:26:40.213466 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e518b7c-885e-411f-a595-061c5ed5ec83 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:40.215259 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 26/53] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:26:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:26:40.220615 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e518b7c-885e-411f-a595-061c5ed5ec83 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:40.268293 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:40.283723 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 28/54] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:26:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 08:26:40.307608 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a058eb85-6576-4610-bef6-8b4c60a3cf32 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:40.320500 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e518b7c-885e-411f-a595-061c5ed5ec83 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:40.354583 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400729e8-c3ec-435f-86e3-c64fe2878179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:40.355328 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400729e8-c3ec-435f-86e3-c64fe2878179 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:40.355622 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400729e8-c3ec-435f-86e3-c64fe2878179 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:40.355946 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400729e8-c3ec-435f-86e3-c64fe2878179 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:40.356905 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400729e8-c3ec-435f-86e3-c64fe2878179 None admin] RBAC: Adding query filter params (role.id=acfe707eee6944ad94d1915fc9ee7ea3, scope.domain.id=default, user.id=268e75b173ff435b8dfbe2517f69c015) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:26:40.357269 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400729e8-c3ec-435f-86e3-c64fe2878179 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:40.359101 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400729e8-c3ec-435f-86e3-c64fe2878179 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:40.367346 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 27/55] 10.222.0.13 () {56 vars in 1322 bytes} [Fri Jun 24 08:26:40 2022] GET /identity/v3/role_assignments?user.id=268e75b173ff435b8dfbe2517f69c015&scope.domain.id=default&role.id=acfe707eee6944ad94d1915fc9ee7ea3 => generated 530 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:26:40.527010 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a058eb85-6576-4610-bef6-8b4c60a3cf32 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:40.564380 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d964b21-8134-4a35-ad00-5dc21f6bd1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:40.565169 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d964b21-8134-4a35-ad00-5dc21f6bd1d9 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:40.565459 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d964b21-8134-4a35-ad00-5dc21f6bd1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:40.565767 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d964b21-8134-4a35-ad00-5dc21f6bd1d9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:40.566946 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d964b21-8134-4a35-ad00-5dc21f6bd1d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:40.568411 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d964b21-8134-4a35-ad00-5dc21f6bd1d9 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:40.570070 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 29/56] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:26:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:26:40.576175 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400729e8-c3ec-435f-86e3-c64fe2878179 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:40.595626 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:40.674645 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400729e8-c3ec-435f-86e3-c64fe2878179 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:40.706423 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c2e1fc-f92c-4515-ae87-467334d27168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:40.707070 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c2e1fc-f92c-4515-ae87-467334d27168 None admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:40.707361 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c2e1fc-f92c-4515-ae87-467334d27168 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:40.707637 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c2e1fc-f92c-4515-ae87-467334d27168 None admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:40.708343 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c2e1fc-f92c-4515-ae87-467334d27168 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:40.709812 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c2e1fc-f92c-4515-ae87-467334d27168 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:40.733605 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 28/57] 10.222.0.13 () {60 vars in 1140 bytes} [Fri Jun 24 08:26:40 2022] POST /identity/v3/projects => generated 298 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:26:42.765294 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef3fc57-a40e-425f-8dd9-9d37e01c3e56 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:42.765844 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef3fc57-a40e-425f-8dd9-9d37e01c3e56 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:42.765951 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef3fc57-a40e-425f-8dd9-9d37e01c3e56 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:42.767278 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 30/58] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:26:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:26:42.772508 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f023c-8258-4afe-a1b6-d36619e86758 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:42.772898 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f023c-8258-4afe-a1b6-d36619e86758 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:42.773076 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f023c-8258-4afe-a1b6-d36619e86758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:42.792415 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-519f023c-8258-4afe-a1b6-d36619e86758 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:26:42.793634 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-519f023c-8258-4afe-a1b6-d36619e86758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:26:42.855755 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:42.861538 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 29/59] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:26:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:26:42.888099 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef3fc57-a40e-425f-8dd9-9d37e01c3e56 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:42.944824 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:42.989287 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef3fc57-a40e-425f-8dd9-9d37e01c3e56 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:43.021935 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd648fa2-cafa-4032-b3dd-fe75d4c20262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:43.022654 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd648fa2-cafa-4032-b3dd-fe75d4c20262 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:43.022940 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd648fa2-cafa-4032-b3dd-fe75d4c20262 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:43.023215 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd648fa2-cafa-4032-b3dd-fe75d4c20262 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:43.025116 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd648fa2-cafa-4032-b3dd-fe75d4c20262 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:43.026580 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd648fa2-cafa-4032-b3dd-fe75d4c20262 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:43.028191 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 31/60] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:26:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:26:43.033705 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f023c-8258-4afe-a1b6-d36619e86758 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:43.134905 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f023c-8258-4afe-a1b6-d36619e86758 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:43.168283 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf71a05-fb2e-4896-93f4-27fad6e66637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:43.168939 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf71a05-fb2e-4896-93f4-27fad6e66637 None admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:43.169229 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf71a05-fb2e-4896-93f4-27fad6e66637 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:43.169511 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf71a05-fb2e-4896-93f4-27fad6e66637 None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:43.170250 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf71a05-fb2e-4896-93f4-27fad6e66637 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:43.171703 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf71a05-fb2e-4896-93f4-27fad6e66637 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:43.196896 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 30/61] 10.222.0.13 () {60 vars in 1132 bytes} [Fri Jun 24 08:26:43 2022] POST /identity/v3/users => generated 275 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:26:46.301560 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7309e3b-6393-4824-943a-cfd85e443a96 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:46.302486 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7309e3b-6393-4824-943a-cfd85e443a96 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:46.302915 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7309e3b-6393-4824-943a-cfd85e443a96 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:46.304448 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 32/62] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:26:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:26:46.310701 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd2a7d-c330-4088-b52d-5f442c74c7c5 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:46.311154 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd2a7d-c330-4088-b52d-5f442c74c7c5 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:46.311261 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd2a7d-c330-4088-b52d-5f442c74c7c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:46.334676 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-13bd2a7d-c330-4088-b52d-5f442c74c7c5 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:26:46.335892 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-13bd2a7d-c330-4088-b52d-5f442c74c7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:26:46.407656 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:46.433353 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 31/63] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:26:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:26:46.474413 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7309e3b-6393-4824-943a-cfd85e443a96 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:46.618748 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7309e3b-6393-4824-943a-cfd85e443a96 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:46.652101 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:46.670472 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e084b2-a207-406c-8b28-17c0bccabe13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:46.671297 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e084b2-a207-406c-8b28-17c0bccabe13 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:46.671581 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e084b2-a207-406c-8b28-17c0bccabe13 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:46.671862 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e084b2-a207-406c-8b28-17c0bccabe13 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:46.684245 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e084b2-a207-406c-8b28-17c0bccabe13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:26:46.691888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-d8e084b2-a207-406c-8b28-17c0bccabe13 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:26:46.691888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:26:46.691888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:26:46.691888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:26:46.691888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:26:46.691888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:26:46.691888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:26:46.691888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:26:46.691888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:26:46.691888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:26:46.691888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:26:46.691888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:26:46.691888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:26:46.691888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:26:46.691888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:26:46.691888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:26:46.692720 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:26:46.692720 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:26:46.692720 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:26:46.692720 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:26:46.692720 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:26:46.692720 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:26:46.692720 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:26:46.692720 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:26:46.692720 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:26:46.692720 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:26:46.692720 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:26:46.692720 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:26:46.692720 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:26:46.692720 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:26:46.692720 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:26:46.692720 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:26:46.693490 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:26:46.693490 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:26:46.693490 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:26:46.693490 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:26:46.693490 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:26:46.693490 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:26:46.693490 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:26:46.693490 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:26:46.693490 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:26:46.693490 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:26:46.693490 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:26:46.693490 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:26:46.693490 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 33/64] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:26:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:26:46.702836 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bd2a7d-c330-4088-b52d-5f442c74c7c5 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:46.710836 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1012a-23cb-41a9-b5c0-2bf24c2d843c None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:46.711661 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1012a-23cb-41a9-b5c0-2bf24c2d843c None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:46.711944 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1012a-23cb-41a9-b5c0-2bf24c2d843c None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:46.713542 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 34/65] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:26:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:26:46.722658 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea92db-2575-4ea5-b605-e39ca7c761b0 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:46.722944 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea92db-2575-4ea5-b605-e39ca7c761b0 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:46.723220 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea92db-2575-4ea5-b605-e39ca7c761b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:46.754739 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-52ea92db-2575-4ea5-b605-e39ca7c761b0 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:26:46.755945 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-52ea92db-2575-4ea5-b605-e39ca7c761b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:26:46.831437 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bd2a7d-c330-4088-b52d-5f442c74c7c5 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:46.842481 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 35/66] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:26:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:26:46.844124 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75994806-6500-4d57-935b-a352aad0b3df None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:46.844422 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75994806-6500-4d57-935b-a352aad0b3df None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:46.844707 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75994806-6500-4d57-935b-a352aad0b3df None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:46.845997 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 36/67] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:26:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:26:46.847260 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf5f6c5-f474-41fc-a65b-46ede14ce0a7 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:46.847786 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf5f6c5-f474-41fc-a65b-46ede14ce0a7 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:46.848260 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf5f6c5-f474-41fc-a65b-46ede14ce0a7 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:46.854062 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 37/68] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:26:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:26:46.855493 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7602dc-b2d9-4f23-b4fe-67f755b34a3a None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:46.855986 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7602dc-b2d9-4f23-b4fe-67f755b34a3a None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:46.856484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7602dc-b2d9-4f23-b4fe-67f755b34a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:46.878325 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c1b3b8-59d6-4068-bf87-d6c526bb66fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:46.878927 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c1b3b8-59d6-4068-bf87-d6c526bb66fd None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:46.879213 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c1b3b8-59d6-4068-bf87-d6c526bb66fd None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:46.879513 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c1b3b8-59d6-4068-bf87-d6c526bb66fd None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:46.879768 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-be7602dc-b2d9-4f23-b4fe-67f755b34a3a None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:26:46.880270 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c1b3b8-59d6-4068-bf87-d6c526bb66fd None admin] RBAC: Adding query filter params (name=member) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:26:46.880576 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c1b3b8-59d6-4068-bf87-d6c526bb66fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:46.880986 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-be7602dc-b2d9-4f23-b4fe-67f755b34a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:26:46.881999 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c1b3b8-59d6-4068-bf87-d6c526bb66fd None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:46.891723 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 32/69] 10.222.0.13 () {56 vars in 1101 bytes} [Fri Jun 24 08:26:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:26:46.891723 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea801615-ef10-4509-8f39-5b89ab5f1f19 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:46.891723 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea801615-ef10-4509-8f39-5b89ab5f1f19 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:46.891723 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea801615-ef10-4509-8f39-5b89ab5f1f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:46.909955 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-ea801615-ef10-4509-8f39-5b89ab5f1f19 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:26:46.911138 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-ea801615-ef10-4509-8f39-5b89ab5f1f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:26:46.955931 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:46.958913 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 38/70] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:26:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:26:46.959758 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7602dc-b2d9-4f23-b4fe-67f755b34a3a None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:46.984111 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 33/71] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:26:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:26:46.985171 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea801615-ef10-4509-8f39-5b89ab5f1f19 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:47.005673 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:47.062677 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7602dc-b2d9-4f23-b4fe-67f755b34a3a None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:47.092776 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea801615-ef10-4509-8f39-5b89ab5f1f19 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:47.096185 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737e437d-5840-46db-9fca-add09b965969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:47.096971 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e437d-5840-46db-9fca-add09b965969 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:47.097257 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e437d-5840-46db-9fca-add09b965969 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:47.097533 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e437d-5840-46db-9fca-add09b965969 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:47.108116 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737e437d-5840-46db-9fca-add09b965969 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:26:47.115623 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-737e437d-5840-46db-9fca-add09b965969 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:26:47.115623 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:26:47.115623 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:26:47.115623 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:26:47.115623 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:26:47.115623 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:26:47.115623 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:26:47.115623 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:26:47.115623 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:26:47.115623 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:26:47.115623 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:26:47.115623 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:26:47.115623 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:26:47.115623 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:26:47.115623 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:26:47.115623 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:26:47.116459 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:26:47.116459 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:26:47.116459 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:26:47.116459 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:26:47.116459 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:26:47.116459 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:26:47.116459 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:26:47.116459 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:26:47.116459 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:26:47.116459 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:26:47.116459 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:26:47.116459 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:26:47.116459 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:26:47.116459 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:26:47.116459 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:26:47.116459 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:26:47.117191 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:26:47.117191 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:26:47.117191 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:26:47.117191 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:26:47.117191 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:26:47.117191 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:26:47.117191 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:26:47.117191 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:26:47.117191 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:26:47.117191 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:26:47.117191 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:26:47.117191 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:26:47.117191 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 39/72] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Jun 24 08:26:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:26:47.118666 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d26a7a-3f40-43d0-8a00-bac559cd1433 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:47.118957 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d26a7a-3f40-43d0-8a00-bac559cd1433 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:47.119106 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d26a7a-3f40-43d0-8a00-bac559cd1433 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:47.120373 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 40/73] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:26:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:26:47.121054 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d26a7a-3f40-43d0-8a00-bac559cd1433 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:47.131970 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3696480-ee44-4673-b87f-0432f8587d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:47.132775 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3696480-ee44-4673-b87f-0432f8587d8e None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:47.133125 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3696480-ee44-4673-b87f-0432f8587d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:47.133403 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3696480-ee44-4673-b87f-0432f8587d8e None admin] PATH_INFO: `/v3/users/8e322e2cfd9e4eca9f58d562b4da2dad` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:47.148691 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3696480-ee44-4673-b87f-0432f8587d8e None admin] RBAC: Authorizing `identity:get_user(user_id=8e322e2cfd9e4eca9f58d562b4da2dad)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:47.150649 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3696480-ee44-4673-b87f-0432f8587d8e None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:47.152400 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 34/74] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Jun 24 08:26:46 2022] GET /identity/v3/users/8e322e2cfd9e4eca9f58d562b4da2dad => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:26:47.157654 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3696480-ee44-4673-b87f-0432f8587d8e None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:47.197542 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:47.236208 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d26a7a-3f40-43d0-8a00-bac559cd1433 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:47.238908 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:47.265212 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3696480-ee44-4673-b87f-0432f8587d8e None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:47.270887 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c183d1c3-c6bf-4094-8d1b-4ff1e877e92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:47.271703 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c183d1c3-c6bf-4094-8d1b-4ff1e877e92c None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:47.271993 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c183d1c3-c6bf-4094-8d1b-4ff1e877e92c None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:47.272281 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c183d1c3-c6bf-4094-8d1b-4ff1e877e92c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:47.282961 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c183d1c3-c6bf-4094-8d1b-4ff1e877e92c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:26:47.291794 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-c183d1c3-c6bf-4094-8d1b-4ff1e877e92c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 08:26:47.291794 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:26:47.291794 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:26:47.291794 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:26:47.291794 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:26:47.291794 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:26:47.291794 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:26:47.291794 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:26:47.291794 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:26:47.291794 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:26:47.291794 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:26:47.291794 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:26:47.291794 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:26:47.291794 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:26:47.291794 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:26:47.291794 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:26:47.292647 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:26:47.292647 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:26:47.292647 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:26:47.292647 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:26:47.292647 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:26:47.292647 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:26:47.292647 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:26:47.292647 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:26:47.292647 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:26:47.292647 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:26:47.292647 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:26:47.292647 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:26:47.292647 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:26:47.292647 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:26:47.292647 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:26:47.292647 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:26:47.293380 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:26:47.293380 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:26:47.293380 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:26:47.293380 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:26:47.293380 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:26:47.293380 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:26:47.293380 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:26:47.293380 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:26:47.293380 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:26:47.293380 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:26:47.293380 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 08:26:47.293380 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:26:47.293380 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 41/75] 10.222.0.13 () {56 vars in 1114 bytes} [Fri Jun 24 08:26:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:26:47.294129 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c183d1c3-c6bf-4094-8d1b-4ff1e877e92c None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:47.303400 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8886a1ff-c474-4f81-abe5-c27ce323871f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:47.304177 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8886a1ff-c474-4f81-abe5-c27ce323871f None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:47.304462 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8886a1ff-c474-4f81-abe5-c27ce323871f None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:47.304737 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8886a1ff-c474-4f81-abe5-c27ce323871f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:47.316251 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8886a1ff-c474-4f81-abe5-c27ce323871f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:26:47.323995 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-8886a1ff-c474-4f81-abe5-c27ce323871f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:26:47.323995 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:26:47.323995 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:26:47.323995 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:26:47.323995 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:26:47.323995 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:26:47.323995 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:26:47.323995 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:26:47.323995 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:26:47.323995 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:26:47.323995 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:26:47.323995 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:26:47.323995 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:26:47.323995 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:26:47.323995 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:26:47.323995 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:26:47.324870 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:26:47.324870 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:26:47.324870 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:26:47.324870 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:26:47.324870 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:26:47.324870 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:26:47.324870 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:26:47.324870 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:26:47.324870 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:26:47.324870 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:26:47.324870 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:26:47.324870 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:26:47.324870 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:26:47.324870 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:26:47.324870 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:26:47.324870 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:26:47.325599 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:26:47.325599 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:26:47.325599 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:26:47.325599 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:26:47.325599 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:26:47.325599 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:26:47.325599 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:26:47.325599 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:26:47.325599 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:26:47.325599 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:26:47.325599 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:26:47.325599 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:26:47.325599 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 35/76] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:26:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:26:47.326425 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a0c5e-4fd0-4071-aa7e-10d1f3871ae0 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:47.326792 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a0c5e-4fd0-4071-aa7e-10d1f3871ae0 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:47.327017 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a0c5e-4fd0-4071-aa7e-10d1f3871ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:47.343716 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-720a0c5e-4fd0-4071-aa7e-10d1f3871ae0 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:26:47.344907 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-720a0c5e-4fd0-4071-aa7e-10d1f3871ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:26:47.405971 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c183d1c3-c6bf-4094-8d1b-4ff1e877e92c None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:47.410832 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 36/77] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:26:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:26:47.411670 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720a0c5e-4fd0-4071-aa7e-10d1f3871ae0 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:47.440837 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8442f2a7-7a99-4850-b29b-4fe46b0914e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:47.441499 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8442f2a7-7a99-4850-b29b-4fe46b0914e9 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:47.441812 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8442f2a7-7a99-4850-b29b-4fe46b0914e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:47.442087 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8442f2a7-7a99-4850-b29b-4fe46b0914e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:47.442830 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8442f2a7-7a99-4850-b29b-4fe46b0914e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:26:47.443137 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8442f2a7-7a99-4850-b29b-4fe46b0914e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:47.444517 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8442f2a7-7a99-4850-b29b-4fe46b0914e9 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:47.448165 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:47.453458 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 42/78] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:26:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:26:47.454273 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8442f2a7-7a99-4850-b29b-4fe46b0914e9 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:47.484572 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:47.545876 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720a0c5e-4fd0-4071-aa7e-10d1f3871ae0 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:47.558503 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8442f2a7-7a99-4850-b29b-4fe46b0914e9 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:47.578751 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce79509-d10c-4189-b2bd-83919dc8ee29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:47.579394 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce79509-d10c-4189-b2bd-83919dc8ee29 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:47.579680 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce79509-d10c-4189-b2bd-83919dc8ee29 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:47.579959 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce79509-d10c-4189-b2bd-83919dc8ee29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:47.592061 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f3cd0-2eaf-4b3a-a0b7-11407b746d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:47.592731 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f3cd0-2eaf-4b3a-a0b7-11407b746d0d None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:47.593023 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f3cd0-2eaf-4b3a-a0b7-11407b746d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:47.593305 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f3cd0-2eaf-4b3a-a0b7-11407b746d0d None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:47.594094 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2f3cd0-2eaf-4b3a-a0b7-11407b746d0d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:26:47.594406 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2f3cd0-2eaf-4b3a-a0b7-11407b746d0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:47.595787 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2f3cd0-2eaf-4b3a-a0b7-11407b746d0d None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:47.604232 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 43/79] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:26:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:26:47.604902 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f3cd0-2eaf-4b3a-a0b7-11407b746d0d None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:47.651366 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce79509-d10c-4189-b2bd-83919dc8ee29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:47.652869 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce79509-d10c-4189-b2bd-83919dc8ee29 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:47.678160 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:47.695016 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:47.708103 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f3cd0-2eaf-4b3a-a0b7-11407b746d0d None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:47.740888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9dee7f-023f-4361-ae7a-bf6076bd0542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:47.741555 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9dee7f-023f-4361-ae7a-bf6076bd0542 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:47.741886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9dee7f-023f-4361-ae7a-bf6076bd0542 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:47.742169 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9dee7f-023f-4361-ae7a-bf6076bd0542 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:47.743539 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9dee7f-023f-4361-ae7a-bf6076bd0542 None admin] RBAC: Adding query filter params (name=member) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:26:47.743918 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9dee7f-023f-4361-ae7a-bf6076bd0542 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:47.745312 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9dee7f-023f-4361-ae7a-bf6076bd0542 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:47.752807 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 37/80] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:26:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:26:47.757957 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 44/81] 10.222.0.13 () {56 vars in 1101 bytes} [Fri Jun 24 08:26:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:26:47.758159 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce79509-d10c-4189-b2bd-83919dc8ee29 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:47.758311 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9dee7f-023f-4361-ae7a-bf6076bd0542 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:47.853084 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce79509-d10c-4189-b2bd-83919dc8ee29 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:47.856878 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9dee7f-023f-4361-ae7a-bf6076bd0542 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:47.876578 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:47.888818 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd220fa-4c0e-4e29-963a-22ca53560439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:47.889622 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd220fa-4c0e-4e29-963a-22ca53560439 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:47.889950 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd220fa-4c0e-4e29-963a-22ca53560439 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:47.890287 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd220fa-4c0e-4e29-963a-22ca53560439 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:47.891503 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd220fa-4c0e-4e29-963a-22ca53560439 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:47.892313 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a22d9f-d599-472d-98f6-d32fe9b8969d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:47.892974 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd220fa-4c0e-4e29-963a-22ca53560439 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:47.893186 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a22d9f-d599-472d-98f6-d32fe9b8969d None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:47.893473 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a22d9f-d599-472d-98f6-d32fe9b8969d None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:47.893784 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a22d9f-d599-472d-98f6-d32fe9b8969d None admin] PATH_INFO: `/v3/users/268e75b173ff435b8dfbe2517f69c015` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:47.894814 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 38/82] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:26:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:26:47.895063 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a22d9f-d599-472d-98f6-d32fe9b8969d None admin] RBAC: Authorizing `identity:get_user(user_id=268e75b173ff435b8dfbe2517f69c015)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:47.895395 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd220fa-4c0e-4e29-963a-22ca53560439 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:47.896576 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a22d9f-d599-472d-98f6-d32fe9b8969d None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:47.903616 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 45/83] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Jun 24 08:26:47 2022] GET /identity/v3/users/268e75b173ff435b8dfbe2517f69c015 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:26:47.903616 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a22d9f-d599-472d-98f6-d32fe9b8969d None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:47.918787 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:48.008245 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a22d9f-d599-472d-98f6-d32fe9b8969d None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:48.013269 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd220fa-4c0e-4e29-963a-22ca53560439 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:48.047517 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9927ccc1-5449-4969-b602-875aee067ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:48.050213 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9255b8-555a-4dfe-a175-700f8b88b2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:48.050327 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9927ccc1-5449-4969-b602-875aee067ae7 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:48.050327 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9927ccc1-5449-4969-b602-875aee067ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:48.050327 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9927ccc1-5449-4969-b602-875aee067ae7 None admin] PATH_INFO: `/v3/users/8e322e2cfd9e4eca9f58d562b4da2dad` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:48.050570 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9255b8-555a-4dfe-a175-700f8b88b2ec None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:48.050570 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9255b8-555a-4dfe-a175-700f8b88b2ec None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:48.050570 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9255b8-555a-4dfe-a175-700f8b88b2ec None admin] PATH_INFO: `/v3/projects/0a1cff337cd248ba94724654a9527e19` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:48.050849 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9927ccc1-5449-4969-b602-875aee067ae7 None admin] RBAC: Authorizing `identity:get_user(user_id=8e322e2cfd9e4eca9f58d562b4da2dad)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:48.051684 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9255b8-555a-4dfe-a175-700f8b88b2ec None admin] RBAC: Authorizing `identity:get_project(project_id=0a1cff337cd248ba94724654a9527e19)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:48.052485 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9927ccc1-5449-4969-b602-875aee067ae7 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:48.053128 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9255b8-555a-4dfe-a175-700f8b88b2ec None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:48.055457 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 46/84] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:26:47 2022] GET /identity/v3/projects/0a1cff337cd248ba94724654a9527e19 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:26:48.060475 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 39/85] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Jun 24 08:26:47 2022] GET /identity/v3/users/8e322e2cfd9e4eca9f58d562b4da2dad => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:26:48.060610 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9255b8-555a-4dfe-a175-700f8b88b2ec None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:48.060697 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9927ccc1-5449-4969-b602-875aee067ae7 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:48.128755 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:48.159067 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:48.176246 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9255b8-555a-4dfe-a175-700f8b88b2ec None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:48.177611 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9927ccc1-5449-4969-b602-875aee067ae7 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:48.215100 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53375252-e4ea-4cff-b127-9d354f600afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:48.215758 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53375252-e4ea-4cff-b127-9d354f600afc None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:48.216795 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53375252-e4ea-4cff-b127-9d354f600afc None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:48.216795 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53375252-e4ea-4cff-b127-9d354f600afc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:48.219247 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255f7287-1744-4b1e-8bf5-6fd171169adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:48.220278 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f7287-1744-4b1e-8bf5-6fd171169adf None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:48.220583 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f7287-1744-4b1e-8bf5-6fd171169adf None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:48.220862 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f7287-1744-4b1e-8bf5-6fd171169adf None admin] PATH_INFO: `/v3/users/8e322e2cfd9e4eca9f58d562b4da2dad` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:48.222800 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255f7287-1744-4b1e-8bf5-6fd171169adf None admin] RBAC: Authorizing `identity:get_user(user_id=8e322e2cfd9e4eca9f58d562b4da2dad)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:48.224374 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255f7287-1744-4b1e-8bf5-6fd171169adf None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:48.226287 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 47/86] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Jun 24 08:26:48 2022] GET /identity/v3/users/8e322e2cfd9e4eca9f58d562b4da2dad => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:26:48.227395 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255f7287-1744-4b1e-8bf5-6fd171169adf None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:48.296419 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53375252-e4ea-4cff-b127-9d354f600afc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:48.297840 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53375252-e4ea-4cff-b127-9d354f600afc None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:48.343989 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255f7287-1744-4b1e-8bf5-6fd171169adf None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:48.359651 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:48.388311 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd66187f-0966-4594-86da-f70a1b3ee070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:48.389000 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd66187f-0966-4594-86da-f70a1b3ee070 None admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:48.389399 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd66187f-0966-4594-86da-f70a1b3ee070 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:48.389965 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd66187f-0966-4594-86da-f70a1b3ee070 None admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:48.390895 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd66187f-0966-4594-86da-f70a1b3ee070 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:48.392861 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd66187f-0966-4594-86da-f70a1b3ee070 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:48.407388 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 40/87] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:26:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:26:48.408242 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53375252-e4ea-4cff-b127-9d354f600afc None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:48.423950 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 48/88] 10.222.0.13 () {60 vars in 1132 bytes} [Fri Jun 24 08:26:48 2022] POST /identity/v3/users => generated 289 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:26:48.424882 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd66187f-0966-4594-86da-f70a1b3ee070 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:48.445976 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:48.561315 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53375252-e4ea-4cff-b127-9d354f600afc None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:48.562577 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd66187f-0966-4594-86da-f70a1b3ee070 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:48.608678 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d723f25d-bc97-4373-9f9e-4d5e27d54dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:48.610374 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d723f25d-bc97-4373-9f9e-4d5e27d54dcd None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:48.610673 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d723f25d-bc97-4373-9f9e-4d5e27d54dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:48.610952 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d723f25d-bc97-4373-9f9e-4d5e27d54dcd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:48.611099 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed78c3d6-6737-403f-869e-f3ace4b39164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:48.611720 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed78c3d6-6737-403f-869e-f3ace4b39164 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:48.612008 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed78c3d6-6737-403f-869e-f3ace4b39164 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:48.612284 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed78c3d6-6737-403f-869e-f3ace4b39164 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:48.613210 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed78c3d6-6737-403f-869e-f3ace4b39164 None admin] RBAC: Adding query filter params (role.id=bd56838b71ec4a5781fe2df9f3f4f1ab, scope.project.id=0a1cff337cd248ba94724654a9527e19, user.id=8e322e2cfd9e4eca9f58d562b4da2dad) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:26:48.614236 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed78c3d6-6737-403f-869e-f3ace4b39164 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:48.615026 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed78c3d6-6737-403f-869e-f3ace4b39164 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:48.635260 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 49/89] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Jun 24 08:26:48 2022] GET /identity/v3/role_assignments?user.id=8e322e2cfd9e4eca9f58d562b4da2dad&scope.project.id=0a1cff337cd248ba94724654a9527e19&role.id=bd56838b71ec4a5781fe2df9f3f4f1ab => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:26:48.636793 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed78c3d6-6737-403f-869e-f3ace4b39164 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:48.656813 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:48.722854 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d723f25d-bc97-4373-9f9e-4d5e27d54dcd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:48.724809 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d723f25d-bc97-4373-9f9e-4d5e27d54dcd None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:48.759595 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:48.800084 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed78c3d6-6737-403f-869e-f3ace4b39164 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:48.857273 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6adaae6-a6f4-419c-ae69-6923a1b58237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:48.857879 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6adaae6-a6f4-419c-ae69-6923a1b58237 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:48.858168 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6adaae6-a6f4-419c-ae69-6923a1b58237 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:48.858445 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6adaae6-a6f4-419c-ae69-6923a1b58237 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:48.911223 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 41/90] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:26:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:26:48.913565 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d723f25d-bc97-4373-9f9e-4d5e27d54dcd None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:48.960126 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6adaae6-a6f4-419c-ae69-6923a1b58237 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:48.961919 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6adaae6-a6f4-419c-ae69-6923a1b58237 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:48.988906 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:49.038896 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d723f25d-bc97-4373-9f9e-4d5e27d54dcd None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:49.080122 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2669740-7e1e-4c43-a8c5-5226f87fa263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:49.081327 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2669740-7e1e-4c43-a8c5-5226f87fa263 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:49.081875 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2669740-7e1e-4c43-a8c5-5226f87fa263 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:49.082373 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2669740-7e1e-4c43-a8c5-5226f87fa263 None admin] PATH_INFO: `/v3/projects/0a1cff337cd248ba94724654a9527e19` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:49.083784 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2669740-7e1e-4c43-a8c5-5226f87fa263 None admin] RBAC: Authorizing `identity:get_project(project_id=0a1cff337cd248ba94724654a9527e19)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:49.085448 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2669740-7e1e-4c43-a8c5-5226f87fa263 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:49.088076 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 42/91] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:26:48 2022] GET /identity/v3/projects/0a1cff337cd248ba94724654a9527e19 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:26:49.088678 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:49.089237 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2669740-7e1e-4c43-a8c5-5226f87fa263 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:49.122374 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 50/92] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:26:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:26:49.123396 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6adaae6-a6f4-419c-ae69-6923a1b58237 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:49.208860 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2669740-7e1e-4c43-a8c5-5226f87fa263 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:49.234769 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6adaae6-a6f4-419c-ae69-6923a1b58237 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:49.239142 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:49.249742 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb250e37-1c90-431c-bf3e-7bd617eda288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:49.254775 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb250e37-1c90-431c-bf3e-7bd617eda288 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:49.255305 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb250e37-1c90-431c-bf3e-7bd617eda288 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:49.255806 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb250e37-1c90-431c-bf3e-7bd617eda288 None admin] PATH_INFO: `/v3/projects/0a1cff337cd248ba94724654a9527e19` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:49.257297 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb250e37-1c90-431c-bf3e-7bd617eda288 None admin] RBAC: Authorizing `identity:get_project(project_id=0a1cff337cd248ba94724654a9527e19)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:49.259778 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb250e37-1c90-431c-bf3e-7bd617eda288 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:49.262382 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 43/93] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:26:49 2022] GET /identity/v3/projects/0a1cff337cd248ba94724654a9527e19 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:26:49.263354 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb250e37-1c90-431c-bf3e-7bd617eda288 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:49.275228 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfeb5c4-e712-4a5a-a54e-b5bd5f1ea807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:49.275908 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfeb5c4-e712-4a5a-a54e-b5bd5f1ea807 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:49.276553 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfeb5c4-e712-4a5a-a54e-b5bd5f1ea807 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:49.276553 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfeb5c4-e712-4a5a-a54e-b5bd5f1ea807 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:49.277317 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfeb5c4-e712-4a5a-a54e-b5bd5f1ea807 None admin] RBAC: Adding query filter params (role.id=acfe707eee6944ad94d1915fc9ee7ea3, scope.project.id=0a1cff337cd248ba94724654a9527e19, user.id=268e75b173ff435b8dfbe2517f69c015) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:26:49.277638 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfeb5c4-e712-4a5a-a54e-b5bd5f1ea807 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:49.279153 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfeb5c4-e712-4a5a-a54e-b5bd5f1ea807 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:49.285295 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 51/94] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Jun 24 08:26:49 2022] GET /identity/v3/role_assignments?user.id=268e75b173ff435b8dfbe2517f69c015&scope.project.id=0a1cff337cd248ba94724654a9527e19&role.id=acfe707eee6944ad94d1915fc9ee7ea3 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:26:49.290502 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfeb5c4-e712-4a5a-a54e-b5bd5f1ea807 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:49.361062 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:49.412502 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb250e37-1c90-431c-bf3e-7bd617eda288 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:49.452678 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d37cd5-2838-43d2-8ecd-9a5e8452f3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:49.453567 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d37cd5-2838-43d2-8ecd-9a5e8452f3bb None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:49.453886 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d37cd5-2838-43d2-8ecd-9a5e8452f3bb None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:49.454166 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d37cd5-2838-43d2-8ecd-9a5e8452f3bb None admin] PATH_INFO: `/v3/projects/2ce34355c4f84047a2f4e1d1adcee45f` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:49.455405 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d37cd5-2838-43d2-8ecd-9a5e8452f3bb None admin] RBAC: Authorizing `identity:get_project(project_id=2ce34355c4f84047a2f4e1d1adcee45f)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:49.456889 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d37cd5-2838-43d2-8ecd-9a5e8452f3bb None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:49.458939 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 44/95] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:26:49 2022] GET /identity/v3/projects/2ce34355c4f84047a2f4e1d1adcee45f => generated 312 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:26:49.465966 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d37cd5-2838-43d2-8ecd-9a5e8452f3bb None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:49.532428 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfeb5c4-e712-4a5a-a54e-b5bd5f1ea807 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:49.546170 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:49.593459 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d37cd5-2838-43d2-8ecd-9a5e8452f3bb None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:49.614428 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f16422a-c18a-4471-abea-731fb084fe0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:49.615105 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16422a-c18a-4471-abea-731fb084fe0f None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:49.615390 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16422a-c18a-4471-abea-731fb084fe0f None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:49.615667 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16422a-c18a-4471-abea-731fb084fe0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:49.616519 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f16422a-c18a-4471-abea-731fb084fe0f None admin] RBAC: Adding query filter params (role.id=d6ada62a74a94222b864a6c283f8d9b1, scope.project.id=0a1cff337cd248ba94724654a9527e19, user.id=8e322e2cfd9e4eca9f58d562b4da2dad) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:26:49.616833 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f16422a-c18a-4471-abea-731fb084fe0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:49.619247 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f16422a-c18a-4471-abea-731fb084fe0f None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:49.630856 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 52/96] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Jun 24 08:26:49 2022] GET /identity/v3/role_assignments?user.id=8e322e2cfd9e4eca9f58d562b4da2dad&scope.project.id=0a1cff337cd248ba94724654a9527e19&role.id=d6ada62a74a94222b864a6c283f8d9b1 => generated 259 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:26:49.654080 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a954a6c-b436-4706-a887-ddc55da1693c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:49.654807 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a954a6c-b436-4706-a887-ddc55da1693c None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:49.655096 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a954a6c-b436-4706-a887-ddc55da1693c None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:49.655376 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a954a6c-b436-4706-a887-ddc55da1693c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:49.656290 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a954a6c-b436-4706-a887-ddc55da1693c None admin] RBAC: Adding query filter params (role.id=bd56838b71ec4a5781fe2df9f3f4f1ab, scope.project.id=2ce34355c4f84047a2f4e1d1adcee45f, user.id=8e322e2cfd9e4eca9f58d562b4da2dad) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:26:49.656656 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a954a6c-b436-4706-a887-ddc55da1693c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:49.658216 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a954a6c-b436-4706-a887-ddc55da1693c None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:49.665640 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 45/97] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Jun 24 08:26:49 2022] GET /identity/v3/role_assignments?user.id=8e322e2cfd9e4eca9f58d562b4da2dad&scope.project.id=2ce34355c4f84047a2f4e1d1adcee45f&role.id=bd56838b71ec4a5781fe2df9f3f4f1ab => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:26:51.588068 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd388c9d-15f6-41d9-a09c-11d671cbf145 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:51.588068 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd388c9d-15f6-41d9-a09c-11d671cbf145 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:51.588068 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd388c9d-15f6-41d9-a09c-11d671cbf145 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:51.588068 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 53/98] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:26:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:26:51.598941 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f67cb-8084-4bb1-b05f-a199c7210244 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:51.599771 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f67cb-8084-4bb1-b05f-a199c7210244 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:51.600220 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f67cb-8084-4bb1-b05f-a199c7210244 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:51.601830 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 46/99] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:26:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:26:51.603479 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8715a4d9-c689-44ff-af32-57d1cd6252dd None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:51.605101 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8715a4d9-c689-44ff-af32-57d1cd6252dd None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:51.605560 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8715a4d9-c689-44ff-af32-57d1cd6252dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:51.610965 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd46b7a-7721-456e-9f33-2981804d0218 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:51.611480 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd46b7a-7721-456e-9f33-2981804d0218 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:51.611946 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd46b7a-7721-456e-9f33-2981804d0218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:51.658503 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-7fd46b7a-7721-456e-9f33-2981804d0218 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:26:51.659699 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-8715a4d9-c689-44ff-af32-57d1cd6252dd None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:26:51.661085 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-7fd46b7a-7721-456e-9f33-2981804d0218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:26:51.662459 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-8715a4d9-c689-44ff-af32-57d1cd6252dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:26:51.795002 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 47/100] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:26:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 08:26:51.802187 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 54/101] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:26:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:26:51.803848 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8339ab-dc72-42a3-8d34-b2ed44de529b None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:51.804344 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8339ab-dc72-42a3-8d34-b2ed44de529b None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:51.805407 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8339ab-dc72-42a3-8d34-b2ed44de529b None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:51.807110 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 55/102] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:26:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:26:51.812701 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a4237-c0c1-4afe-aaf0-dc4bedb69629 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:51.813272 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a4237-c0c1-4afe-aaf0-dc4bedb69629 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:51.813771 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a4237-c0c1-4afe-aaf0-dc4bedb69629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:51.837390 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8339ab-dc72-42a3-8d34-b2ed44de529b None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:51.857990 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-999a4237-c0c1-4afe-aaf0-dc4bedb69629 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:26:51.861027 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-999a4237-c0c1-4afe-aaf0-dc4bedb69629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:26:51.893309 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:51.969008 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:51.999514 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 48/103] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:26:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 08:26:52.000834 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999a4237-c0c1-4afe-aaf0-dc4bedb69629 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:52.032534 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8339ab-dc72-42a3-8d34-b2ed44de529b None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:52.108201 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779865e1-14b5-4765-aab9-aaddc07022c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:52.109636 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779865e1-14b5-4765-aab9-aaddc07022c2 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:52.111516 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779865e1-14b5-4765-aab9-aaddc07022c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:52.112007 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779865e1-14b5-4765-aab9-aaddc07022c2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:52.143756 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779865e1-14b5-4765-aab9-aaddc07022c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:26:52.164024 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-779865e1-14b5-4765-aab9-aaddc07022c2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:26:52.164024 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:26:52.164024 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:26:52.164024 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:26:52.164024 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:26:52.164024 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:26:52.164024 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:26:52.164024 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:26:52.164024 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:26:52.164024 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:26:52.164024 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:26:52.164024 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:26:52.164024 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:26:52.164024 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:26:52.164024 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:26:52.164024 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:26:52.165778 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:26:52.165778 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:26:52.165778 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:26:52.165778 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:26:52.165778 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:26:52.165778 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:26:52.165778 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:26:52.165778 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:26:52.165778 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:26:52.165778 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:26:52.165778 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:26:52.165778 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:26:52.165778 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:26:52.165778 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:26:52.165778 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:26:52.165778 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:26:52.166528 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:26:52.166528 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:26:52.166528 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:26:52.166528 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:26:52.166528 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:26:52.166528 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:26:52.166528 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:26:52.166528 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:26:52.166528 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:26:52.166528 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:26:52.166528 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:26:52.166528 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:26:52.171049 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 56/104] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:26:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:26:52.174698 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779865e1-14b5-4765-aab9-aaddc07022c2 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:52.228848 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999a4237-c0c1-4afe-aaf0-dc4bedb69629 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:52.310572 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09077ff2-42a4-42a6-9085-285dc5d4e150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:52.313303 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09077ff2-42a4-42a6-9085-285dc5d4e150 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:52.314444 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09077ff2-42a4-42a6-9085-285dc5d4e150 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:52.314918 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09077ff2-42a4-42a6-9085-285dc5d4e150 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:52.319009 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09077ff2-42a4-42a6-9085-285dc5d4e150 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:52.321323 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09077ff2-42a4-42a6-9085-285dc5d4e150 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:52.327911 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 49/105] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:26:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:26:52.328955 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09077ff2-42a4-42a6-9085-285dc5d4e150 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:52.362296 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:52.372403 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779865e1-14b5-4765-aab9-aaddc07022c2 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:52.409828 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ececb6b-d4c1-40f8-90a5-38e5dbd9c867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:52.410984 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ececb6b-d4c1-40f8-90a5-38e5dbd9c867 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:52.411464 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ececb6b-d4c1-40f8-90a5-38e5dbd9c867 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:52.411951 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ececb6b-d4c1-40f8-90a5-38e5dbd9c867 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:52.436394 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ececb6b-d4c1-40f8-90a5-38e5dbd9c867 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:26:52.452391 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-0ececb6b-d4c1-40f8-90a5-38e5dbd9c867 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 08:26:52.452391 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:26:52.452391 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:26:52.452391 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:26:52.452391 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:26:52.452391 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:26:52.452391 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:26:52.452391 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:26:52.452391 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:26:52.452391 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:26:52.452391 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:26:52.452391 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:26:52.452391 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:26:52.452391 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:26:52.452391 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:26:52.452391 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:26:52.453258 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:26:52.453258 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:26:52.453258 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:26:52.453258 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:26:52.453258 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:26:52.453258 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:26:52.453258 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:26:52.453258 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:26:52.453258 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:26:52.453258 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:26:52.453258 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:26:52.453258 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:26:52.453258 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:26:52.453258 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:26:52.453258 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:26:52.453258 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:26:52.454254 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:26:52.454254 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:26:52.454254 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:26:52.454254 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:26:52.454254 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:26:52.454254 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:26:52.454254 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:26:52.454254 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:26:52.454254 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:26:52.454254 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:26:52.454254 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 08:26:52.454254 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:26:52.457879 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 57/106] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:26:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:26:52.459187 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ececb6b-d4c1-40f8-90a5-38e5dbd9c867 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:52.463653 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:52.540362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09077ff2-42a4-42a6-9085-285dc5d4e150 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:52.587095 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474b2a1c-24b6-494e-8d10-fa33114b31e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:52.588414 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b2a1c-24b6-494e-8d10-fa33114b31e6 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:52.588842 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b2a1c-24b6-494e-8d10-fa33114b31e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:52.589255 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b2a1c-24b6-494e-8d10-fa33114b31e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:52.590227 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474b2a1c-24b6-494e-8d10-fa33114b31e6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:26:52.591246 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474b2a1c-24b6-494e-8d10-fa33114b31e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:52.593068 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474b2a1c-24b6-494e-8d10-fa33114b31e6 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:52.604460 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ececb6b-d4c1-40f8-90a5-38e5dbd9c867 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:52.605729 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 50/107] 10.222.0.13 () {56 vars in 1101 bytes} [Fri Jun 24 08:26:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:26:52.606822 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474b2a1c-24b6-494e-8d10-fa33114b31e6 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:52.643164 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee7f9e0-3135-4e78-9660-5e4e61cfbe83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:52.644169 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7f9e0-3135-4e78-9660-5e4e61cfbe83 None admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:52.644673 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7f9e0-3135-4e78-9660-5e4e61cfbe83 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:52.645148 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7f9e0-3135-4e78-9660-5e4e61cfbe83 None admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:52.646244 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee7f9e0-3135-4e78-9660-5e4e61cfbe83 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:52.647898 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee7f9e0-3135-4e78-9660-5e4e61cfbe83 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:52.675949 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 58/108] 10.222.0.13 () {60 vars in 1141 bytes} [Fri Jun 24 08:26:52 2022] POST /identity/v3/projects => generated 302 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:26:52.677257 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a95c25-1109-417c-b8a8-6ae640a37a47 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:52.677555 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a95c25-1109-417c-b8a8-6ae640a37a47 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:52.677870 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a95c25-1109-417c-b8a8-6ae640a37a47 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:52.679702 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 59/109] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:26:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:26:52.685395 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a95c25-1109-417c-b8a8-6ae640a37a47 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:52.690532 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:52.745417 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474b2a1c-24b6-494e-8d10-fa33114b31e6 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:52.767734 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:52.788897 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21db4ea0-de7d-4700-8131-643f76f7ff2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:52.790845 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db4ea0-de7d-4700-8131-643f76f7ff2d None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:52.791323 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db4ea0-de7d-4700-8131-643f76f7ff2d None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:52.791741 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db4ea0-de7d-4700-8131-643f76f7ff2d None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:52.792807 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21db4ea0-de7d-4700-8131-643f76f7ff2d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:26:52.793288 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21db4ea0-de7d-4700-8131-643f76f7ff2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:52.794899 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21db4ea0-de7d-4700-8131-643f76f7ff2d None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:52.805512 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 51/110] 10.222.0.13 () {56 vars in 1101 bytes} [Fri Jun 24 08:26:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:26:52.807317 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a95c25-1109-417c-b8a8-6ae640a37a47 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:52.807773 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48151899-e946-464e-9641-8ceb9191565d None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:52.808213 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48151899-e946-464e-9641-8ceb9191565d None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:52.808655 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48151899-e946-464e-9641-8ceb9191565d None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:52.810698 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 52/111] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:26:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:26:52.812070 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d60b2-b1b2-421c-a0a5-1fa5002ee881 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:52.812565 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d60b2-b1b2-421c-a0a5-1fa5002ee881 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:52.813030 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d60b2-b1b2-421c-a0a5-1fa5002ee881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:52.833121 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-4c1d60b2-b1b2-421c-a0a5-1fa5002ee881 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:26:52.834827 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-4c1d60b2-b1b2-421c-a0a5-1fa5002ee881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:26:52.849435 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a50103f-38be-4957-8cb6-0bcd040b802e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:52.850434 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50103f-38be-4957-8cb6-0bcd040b802e None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:52.850718 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50103f-38be-4957-8cb6-0bcd040b802e None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:52.850999 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50103f-38be-4957-8cb6-0bcd040b802e None admin] PATH_INFO: `/v3/users/8e322e2cfd9e4eca9f58d562b4da2dad` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:52.852328 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a50103f-38be-4957-8cb6-0bcd040b802e None admin] RBAC: Authorizing `identity:get_user(user_id=8e322e2cfd9e4eca9f58d562b4da2dad)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:52.854022 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a50103f-38be-4957-8cb6-0bcd040b802e None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:52.855787 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 60/112] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Jun 24 08:26:52 2022] GET /identity/v3/users/8e322e2cfd9e4eca9f58d562b4da2dad => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:26:52.856443 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a50103f-38be-4957-8cb6-0bcd040b802e None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:52.911315 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 53/113] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:26:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:26:52.912682 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e461106-7148-4b6a-a034-6d6c7081d27d None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:52.912974 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e461106-7148-4b6a-a034-6d6c7081d27d None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:52.913249 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e461106-7148-4b6a-a034-6d6c7081d27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:52.932338 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-9e461106-7148-4b6a-a034-6d6c7081d27d None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:26:52.932338 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-9e461106-7148-4b6a-a034-6d6c7081d27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:26:52.972096 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:53.013901 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a50103f-38be-4957-8cb6-0bcd040b802e None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:53.017027 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 54/114] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:26:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:26:53.017314 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e461106-7148-4b6a-a034-6d6c7081d27d None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:53.056024 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef47744-931b-4039-8a26-86379d2f16b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:53.056916 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef47744-931b-4039-8a26-86379d2f16b1 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:53.057202 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef47744-931b-4039-8a26-86379d2f16b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:53.057477 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef47744-931b-4039-8a26-86379d2f16b1 None admin] PATH_INFO: `/v3/users/647a5533a0ab41cab27f26ff9d07ff5b` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:53.068189 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:53.073463 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef47744-931b-4039-8a26-86379d2f16b1 None admin] RBAC: Authorizing `identity:get_user(user_id=647a5533a0ab41cab27f26ff9d07ff5b)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:53.075003 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef47744-931b-4039-8a26-86379d2f16b1 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:53.076876 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 61/115] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Jun 24 08:26:52 2022] GET /identity/v3/users/647a5533a0ab41cab27f26ff9d07ff5b => generated 289 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:26:53.077560 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef47744-931b-4039-8a26-86379d2f16b1 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:53.136347 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e461106-7148-4b6a-a034-6d6c7081d27d None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:53.177315 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451fde6a-809a-410d-9349-7d78639ddcc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:53.178218 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451fde6a-809a-410d-9349-7d78639ddcc0 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:53.178505 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451fde6a-809a-410d-9349-7d78639ddcc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:53.178786 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451fde6a-809a-410d-9349-7d78639ddcc0 None admin] PATH_INFO: `/v3/projects/0a1cff337cd248ba94724654a9527e19` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:53.180589 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451fde6a-809a-410d-9349-7d78639ddcc0 None admin] RBAC: Authorizing `identity:get_project(project_id=0a1cff337cd248ba94724654a9527e19)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:53.182104 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451fde6a-809a-410d-9349-7d78639ddcc0 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:53.183833 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 55/116] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:26:53 2022] GET /identity/v3/projects/0a1cff337cd248ba94724654a9527e19 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:26:53.185022 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0183c00-d442-4ec2-a7b5-f0694a176a68 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:53.185313 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0183c00-d442-4ec2-a7b5-f0694a176a68 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:53.185596 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0183c00-d442-4ec2-a7b5-f0694a176a68 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:53.187033 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 56/117] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:26:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:26:53.190460 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0183c00-d442-4ec2-a7b5-f0694a176a68 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:53.194166 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef47744-931b-4039-8a26-86379d2f16b1 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:53.229526 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:53.236239 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f769397-0e48-4de5-8bd0-9cd412906dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:53.237054 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f769397-0e48-4de5-8bd0-9cd412906dec None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:53.237342 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f769397-0e48-4de5-8bd0-9cd412906dec None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:53.237622 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f769397-0e48-4de5-8bd0-9cd412906dec None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:53.250250 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f769397-0e48-4de5-8bd0-9cd412906dec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:26:53.259484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-7f769397-0e48-4de5-8bd0-9cd412906dec None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 08:26:53.259484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:26:53.259484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:26:53.259484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:26:53.259484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:26:53.259484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:26:53.259484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:26:53.259484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:26:53.259484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:26:53.259484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:26:53.259484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:26:53.259484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:26:53.259484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:26:53.259484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:26:53.259484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:26:53.259484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:26:53.260345 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:26:53.260345 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:26:53.260345 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:26:53.260345 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:26:53.260345 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:26:53.260345 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:26:53.260345 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:26:53.260345 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:26:53.260345 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:26:53.260345 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:26:53.260345 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:26:53.260345 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:26:53.260345 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:26:53.260345 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:26:53.260345 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:26:53.260345 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:26:53.261146 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:26:53.261146 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:26:53.261146 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:26:53.261146 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:26:53.261146 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:26:53.261146 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:26:53.261146 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:26:53.261146 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:26:53.261146 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:26:53.261146 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:26:53.261146 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 08:26:53.261146 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:26:53.261146 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 62/118] 10.222.0.13 () {56 vars in 1114 bytes} [Fri Jun 24 08:26:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:26:53.262130 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f769397-0e48-4de5-8bd0-9cd412906dec None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:53.312265 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0183c00-d442-4ec2-a7b5-f0694a176a68 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:53.322755 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:53.356796 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3da78d-d4a6-4ecf-9a7a-e13dc04d2da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:53.357948 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3da78d-d4a6-4ecf-9a7a-e13dc04d2da8 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:53.358392 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3da78d-d4a6-4ecf-9a7a-e13dc04d2da8 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:53.358674 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3da78d-d4a6-4ecf-9a7a-e13dc04d2da8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:53.376491 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3da78d-d4a6-4ecf-9a7a-e13dc04d2da8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:26:53.377951 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f769397-0e48-4de5-8bd0-9cd412906dec None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:53.384865 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-ed3da78d-d4a6-4ecf-9a7a-e13dc04d2da8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:26:53.384865 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:26:53.384865 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:26:53.384865 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:26:53.384865 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:26:53.384865 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:26:53.384865 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:26:53.384865 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:26:53.384865 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:26:53.384865 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:26:53.384865 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:26:53.384865 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:26:53.384865 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:26:53.384865 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:26:53.384865 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:26:53.384865 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:26:53.385742 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:26:53.385742 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:26:53.385742 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:26:53.385742 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:26:53.385742 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:26:53.385742 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:26:53.385742 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:26:53.385742 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:26:53.385742 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:26:53.385742 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:26:53.385742 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:26:53.385742 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:26:53.385742 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:26:53.385742 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:26:53.385742 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:26:53.385742 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:26:53.386503 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:26:53.386503 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:26:53.386503 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:26:53.386503 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:26:53.386503 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:26:53.386503 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:26:53.386503 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:26:53.386503 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:26:53.386503 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:26:53.386503 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:26:53.386503 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:26:53.386503 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:26:53.386503 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 57/119] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Jun 24 08:26:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:26:53.387734 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3da78d-d4a6-4ecf-9a7a-e13dc04d2da8 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:53.418449 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9b6f74-b037-441b-a44a-2d02534a22d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:53.419109 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b6f74-b037-441b-a44a-2d02534a22d5 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:53.419397 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b6f74-b037-441b-a44a-2d02534a22d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:53.419679 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b6f74-b037-441b-a44a-2d02534a22d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:53.480951 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:53.493704 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9b6f74-b037-441b-a44a-2d02534a22d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:53.494969 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9b6f74-b037-441b-a44a-2d02534a22d5 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:53.509183 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3da78d-d4a6-4ecf-9a7a-e13dc04d2da8 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:53.553178 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26616b9-b094-4e06-9f55-17b892b80e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:53.554618 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26616b9-b094-4e06-9f55-17b892b80e46 None admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:53.555115 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26616b9-b094-4e06-9f55-17b892b80e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:53.555490 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26616b9-b094-4e06-9f55-17b892b80e46 None admin] PATH_INFO: `/v3/projects/0a1cff337cd248ba94724654a9527e19/users/8e322e2cfd9e4eca9f58d562b4da2dad/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:53.557575 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26616b9-b094-4e06-9f55-17b892b80e46 None admin] RBAC: Authorizing `identity:create_grant(project_id=0a1cff337cd248ba94724654a9527e19, user_id=8e322e2cfd9e4eca9f58d562b4da2dad, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:53.559078 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26616b9-b094-4e06-9f55-17b892b80e46 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:53.578130 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 58/120] 10.222.0.13 () {58 vars in 1439 bytes} [Fri Jun 24 08:26:53 2022] PUT /identity/v3/projects/0a1cff337cd248ba94724654a9527e19/users/8e322e2cfd9e4eca9f58d562b4da2dad/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:26:53.579379 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc672a3-8ada-4283-9df1-cbec42e81ae5 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:53.579668 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc672a3-8ada-4283-9df1-cbec42e81ae5 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:53.579945 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc672a3-8ada-4283-9df1-cbec42e81ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:53.601169 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-6fc672a3-8ada-4283-9df1-cbec42e81ae5 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:26:53.603230 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-6fc672a3-8ada-4283-9df1-cbec42e81ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:26:53.625141 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 63/121] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:26:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:26:53.630422 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9b6f74-b037-441b-a44a-2d02534a22d5 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:53.634995 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:53.687620 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 59/122] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:26:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:26:53.689019 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc672a3-8ada-4283-9df1-cbec42e81ae5 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:53.735400 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:53.747571 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9b6f74-b037-441b-a44a-2d02534a22d5 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:53.790826 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f28fc-4190-42d1-afea-67b2d9ab7bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:53.791501 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f28fc-4190-42d1-afea-67b2d9ab7bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:53.791791 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f28fc-4190-42d1-afea-67b2d9ab7bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:53.792067 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f28fc-4190-42d1-afea-67b2d9ab7bc3 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:53.792819 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484f28fc-4190-42d1-afea-67b2d9ab7bc3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:26:53.793129 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484f28fc-4190-42d1-afea-67b2d9ab7bc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:53.794552 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484f28fc-4190-42d1-afea-67b2d9ab7bc3 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:53.802826 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 64/123] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:26:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:26:53.803682 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f28fc-4190-42d1-afea-67b2d9ab7bc3 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:53.805095 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc672a3-8ada-4283-9df1-cbec42e81ae5 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:53.841954 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c85b165-078a-4e11-afa7-4bf6bf0495c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:53.842545 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c85b165-078a-4e11-afa7-4bf6bf0495c8 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:53.842838 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c85b165-078a-4e11-afa7-4bf6bf0495c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:53.843112 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c85b165-078a-4e11-afa7-4bf6bf0495c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:53.843861 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c85b165-078a-4e11-afa7-4bf6bf0495c8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:26:53.844166 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c85b165-078a-4e11-afa7-4bf6bf0495c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:53.845682 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c85b165-078a-4e11-afa7-4bf6bf0495c8 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:53.854716 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 60/124] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:26:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:26:53.855543 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c85b165-078a-4e11-afa7-4bf6bf0495c8 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:53.879370 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:53.934343 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f28fc-4190-42d1-afea-67b2d9ab7bc3 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:53.967267 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c85b165-078a-4e11-afa7-4bf6bf0495c8 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:53.973058 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c8251-05dc-47f3-9322-94825663b31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:53.973992 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c8251-05dc-47f3-9322-94825663b31e None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:53.974254 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c8251-05dc-47f3-9322-94825663b31e None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:53.974534 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c8251-05dc-47f3-9322-94825663b31e None admin] PATH_INFO: `/v3/projects/0a1cff337cd248ba94724654a9527e19` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:53.975766 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8c8251-05dc-47f3-9322-94825663b31e None admin] RBAC: Authorizing `identity:get_project(project_id=0a1cff337cd248ba94724654a9527e19)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:53.977527 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8c8251-05dc-47f3-9322-94825663b31e None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:53.979314 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 65/125] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:26:53 2022] GET /identity/v3/projects/0a1cff337cd248ba94724654a9527e19 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:26:53.980084 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c8251-05dc-47f3-9322-94825663b31e None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:54.003748 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4245f0-2012-4d43-a91e-861ab9e25613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:54.003945 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:54.004910 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4245f0-2012-4d43-a91e-861ab9e25613 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:54.005200 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4245f0-2012-4d43-a91e-861ab9e25613 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:54.005478 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4245f0-2012-4d43-a91e-861ab9e25613 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:54.017108 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4245f0-2012-4d43-a91e-861ab9e25613 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:26:54.025661 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-7d4245f0-2012-4d43-a91e-861ab9e25613 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:26:54.025661 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:26:54.025661 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:26:54.025661 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:26:54.025661 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:26:54.025661 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:26:54.025661 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:26:54.025661 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:26:54.025661 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:26:54.025661 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:26:54.025661 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:26:54.025661 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:26:54.025661 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:26:54.025661 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:26:54.025661 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:26:54.025661 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:26:54.026579 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:26:54.026579 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:26:54.026579 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:26:54.026579 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:26:54.026579 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:26:54.026579 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:26:54.026579 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:26:54.026579 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:26:54.026579 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:26:54.026579 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:26:54.026579 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:26:54.026579 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:26:54.026579 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:26:54.026579 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:26:54.026579 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:26:54.026579 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:26:54.027326 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:26:54.027326 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:26:54.027326 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:26:54.027326 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:26:54.027326 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:26:54.027326 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:26:54.027326 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:26:54.027326 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:26:54.027326 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:26:54.027326 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:26:54.027326 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:26:54.027326 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:26:54.027326 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 61/126] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:26:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:26:54.028441 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4245f0-2012-4d43-a91e-861ab9e25613 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:54.105555 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c8251-05dc-47f3-9322-94825663b31e None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:54.131510 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:54.146212 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e875b1-16ff-48c9-8b0a-de96c872da76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:54.147359 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e875b1-16ff-48c9-8b0a-de96c872da76 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:54.147780 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e875b1-16ff-48c9-8b0a-de96c872da76 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:54.148195 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e875b1-16ff-48c9-8b0a-de96c872da76 None admin] PATH_INFO: `/v3/users/8e322e2cfd9e4eca9f58d562b4da2dad` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:54.149958 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e875b1-16ff-48c9-8b0a-de96c872da76 None admin] RBAC: Authorizing `identity:get_user(user_id=8e322e2cfd9e4eca9f58d562b4da2dad)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:54.151782 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e875b1-16ff-48c9-8b0a-de96c872da76 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:54.153672 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 66/127] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Jun 24 08:26:53 2022] GET /identity/v3/users/8e322e2cfd9e4eca9f58d562b4da2dad => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:26:54.154654 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e875b1-16ff-48c9-8b0a-de96c872da76 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:54.155441 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4245f0-2012-4d43-a91e-861ab9e25613 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:54.199949 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a21639a-de44-4b23-9e3d-f69ec6b07c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:54.200827 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a21639a-de44-4b23-9e3d-f69ec6b07c3a None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:54.201117 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a21639a-de44-4b23-9e3d-f69ec6b07c3a None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:54.201399 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a21639a-de44-4b23-9e3d-f69ec6b07c3a None admin] PATH_INFO: `/v3/users/268e75b173ff435b8dfbe2517f69c015` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:54.202657 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a21639a-de44-4b23-9e3d-f69ec6b07c3a None admin] RBAC: Authorizing `identity:get_user(user_id=268e75b173ff435b8dfbe2517f69c015)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:54.204645 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a21639a-de44-4b23-9e3d-f69ec6b07c3a None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:54.207119 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 62/128] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Jun 24 08:26:54 2022] GET /identity/v3/users/268e75b173ff435b8dfbe2517f69c015 => generated 247 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:26:54.207903 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a21639a-de44-4b23-9e3d-f69ec6b07c3a None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:54.268880 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:54.285462 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e875b1-16ff-48c9-8b0a-de96c872da76 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:54.332363 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a21639a-de44-4b23-9e3d-f69ec6b07c3a None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:54.332583 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee646ea-6c01-4c24-b18f-03078699ccda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:54.333205 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee646ea-6c01-4c24-b18f-03078699ccda None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:54.333500 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee646ea-6c01-4c24-b18f-03078699ccda None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:54.333828 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee646ea-6c01-4c24-b18f-03078699ccda None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:54.335880 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee646ea-6c01-4c24-b18f-03078699ccda None admin] RBAC: Adding query filter params (role.id=986a50f3abf14fdab55012a8e52719c7, scope.project.id=0a1cff337cd248ba94724654a9527e19, user.id=647a5533a0ab41cab27f26ff9d07ff5b) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:26:54.336251 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee646ea-6c01-4c24-b18f-03078699ccda None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:54.337768 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee646ea-6c01-4c24-b18f-03078699ccda None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:54.346038 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 67/129] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Jun 24 08:26:54 2022] GET /identity/v3/role_assignments?user.id=647a5533a0ab41cab27f26ff9d07ff5b&scope.project.id=0a1cff337cd248ba94724654a9527e19&role.id=986a50f3abf14fdab55012a8e52719c7 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:26:54.346879 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee646ea-6c01-4c24-b18f-03078699ccda None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:54.418601 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dc586a-58c6-46f9-ac6f-aab0cc2c5590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:54.419296 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc586a-58c6-46f9-ac6f-aab0cc2c5590 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:54.419587 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc586a-58c6-46f9-ac6f-aab0cc2c5590 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:54.419868 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc586a-58c6-46f9-ac6f-aab0cc2c5590 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:54.420705 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dc586a-58c6-46f9-ac6f-aab0cc2c5590 None admin] RBAC: Adding query filter params (name=member) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:26:54.421015 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dc586a-58c6-46f9-ac6f-aab0cc2c5590 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:54.422526 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dc586a-58c6-46f9-ac6f-aab0cc2c5590 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:54.446169 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 63/130] 10.222.0.13 () {56 vars in 1101 bytes} [Fri Jun 24 08:26:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:26:54.447528 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dc586a-58c6-46f9-ac6f-aab0cc2c5590 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:54.475681 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:54.484207 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee646ea-6c01-4c24-b18f-03078699ccda None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:54.527026 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff507f-4fe6-4f93-b3c0-957b8a9a05c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:54.528086 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff507f-4fe6-4f93-b3c0-957b8a9a05c6 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:54.528384 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff507f-4fe6-4f93-b3c0-957b8a9a05c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:54.528668 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff507f-4fe6-4f93-b3c0-957b8a9a05c6 None admin] PATH_INFO: `/v3/projects/0a1cff337cd248ba94724654a9527e19` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:54.529969 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff507f-4fe6-4f93-b3c0-957b8a9a05c6 None admin] RBAC: Authorizing `identity:get_project(project_id=0a1cff337cd248ba94724654a9527e19)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:54.531469 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff507f-4fe6-4f93-b3c0-957b8a9a05c6 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:54.533279 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 68/131] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:26:54 2022] GET /identity/v3/projects/0a1cff337cd248ba94724654a9527e19 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:26:54.534111 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff507f-4fe6-4f93-b3c0-957b8a9a05c6 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:54.563860 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:54.611442 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dc586a-58c6-46f9-ac6f-aab0cc2c5590 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:54.669529 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c7a23-efc7-48df-9cc6-ad3d6609af9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:54.670478 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c7a23-efc7-48df-9cc6-ad3d6609af9d None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:54.670741 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c7a23-efc7-48df-9cc6-ad3d6609af9d None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:54.671029 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c7a23-efc7-48df-9cc6-ad3d6609af9d None admin] PATH_INFO: `/v3/projects/0a1cff337cd248ba94724654a9527e19` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:54.673107 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67c7a23-efc7-48df-9cc6-ad3d6609af9d None admin] RBAC: Authorizing `identity:get_project(project_id=0a1cff337cd248ba94724654a9527e19)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:54.675687 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67c7a23-efc7-48df-9cc6-ad3d6609af9d None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:54.677595 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 64/132] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:26:54 2022] GET /identity/v3/projects/0a1cff337cd248ba94724654a9527e19 => generated 298 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:26:54.679235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff507f-4fe6-4f93-b3c0-957b8a9a05c6 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:54.682474 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c7a23-efc7-48df-9cc6-ad3d6609af9d None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:54.724294 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8903e952-23ee-4a98-9ae4-5cea6904809b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:54.725559 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8903e952-23ee-4a98-9ae4-5cea6904809b None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:54.726073 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8903e952-23ee-4a98-9ae4-5cea6904809b None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:54.726744 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8903e952-23ee-4a98-9ae4-5cea6904809b None admin] PATH_INFO: `/v3/users/8e322e2cfd9e4eca9f58d562b4da2dad` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:54.728634 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8903e952-23ee-4a98-9ae4-5cea6904809b None admin] RBAC: Authorizing `identity:get_user(user_id=8e322e2cfd9e4eca9f58d562b4da2dad)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:54.730632 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8903e952-23ee-4a98-9ae4-5cea6904809b None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:54.732559 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 69/133] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Jun 24 08:26:54 2022] GET /identity/v3/users/8e322e2cfd9e4eca9f58d562b4da2dad => generated 275 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:26:54.733513 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8903e952-23ee-4a98-9ae4-5cea6904809b None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:54.842179 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:54.857989 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8903e952-23ee-4a98-9ae4-5cea6904809b None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:54.897527 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:54.902114 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006436e5-06e6-4cc6-b00f-4f26b87def96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:54.903395 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006436e5-06e6-4cc6-b00f-4f26b87def96 None admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:54.903883 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006436e5-06e6-4cc6-b00f-4f26b87def96 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:54.904701 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006436e5-06e6-4cc6-b00f-4f26b87def96 None admin] PATH_INFO: `/v3/projects/0a1cff337cd248ba94724654a9527e19/users/268e75b173ff435b8dfbe2517f69c015/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:54.907680 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006436e5-06e6-4cc6-b00f-4f26b87def96 None admin] RBAC: Authorizing `identity:create_grant(project_id=0a1cff337cd248ba94724654a9527e19, user_id=268e75b173ff435b8dfbe2517f69c015, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:54.909458 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006436e5-06e6-4cc6-b00f-4f26b87def96 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:54.929451 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 70/134] 10.222.0.13 () {58 vars in 1439 bytes} [Fri Jun 24 08:26:54 2022] PUT /identity/v3/projects/0a1cff337cd248ba94724654a9527e19/users/268e75b173ff435b8dfbe2517f69c015/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:26:54.930942 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006436e5-06e6-4cc6-b00f-4f26b87def96 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:54.959879 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c7a23-efc7-48df-9cc6-ad3d6609af9d None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:55.044318 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7b9fd5-bb31-4ab2-acf2-12524d574e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:55.045841 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b9fd5-bb31-4ab2-acf2-12524d574e87 None admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:55.046341 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b9fd5-bb31-4ab2-acf2-12524d574e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:55.046821 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b9fd5-bb31-4ab2-acf2-12524d574e87 None admin] PATH_INFO: `/v3/projects/0a1cff337cd248ba94724654a9527e19/users/8e322e2cfd9e4eca9f58d562b4da2dad/roles/d6ada62a74a94222b864a6c283f8d9b1` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:55.049355 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7b9fd5-bb31-4ab2-acf2-12524d574e87 None admin] RBAC: Authorizing `identity:create_grant(project_id=0a1cff337cd248ba94724654a9527e19, user_id=8e322e2cfd9e4eca9f58d562b4da2dad, role_id=d6ada62a74a94222b864a6c283f8d9b1)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:55.051222 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7b9fd5-bb31-4ab2-acf2-12524d574e87 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:55.084111 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 65/135] 10.222.0.13 () {58 vars in 1439 bytes} [Fri Jun 24 08:26:54 2022] PUT /identity/v3/projects/0a1cff337cd248ba94724654a9527e19/users/8e322e2cfd9e4eca9f58d562b4da2dad/roles/d6ada62a74a94222b864a6c283f8d9b1 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:26:55.108488 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8352f8-f65d-476a-a368-5cfb37e665c9 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:55.109006 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8352f8-f65d-476a-a368-5cfb37e665c9 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:55.109289 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8352f8-f65d-476a-a368-5cfb37e665c9 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:55.110829 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 66/136] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:26:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:26:55.118006 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7af2e-86ba-4daa-865b-ad6bdf25128c None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:55.118731 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7af2e-86ba-4daa-865b-ad6bdf25128c None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:55.119229 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7af2e-86ba-4daa-865b-ad6bdf25128c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:55.158898 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-f0f7af2e-86ba-4daa-865b-ad6bdf25128c None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:26:55.159717 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006436e5-06e6-4cc6-b00f-4f26b87def96 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:55.160737 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-f0f7af2e-86ba-4daa-865b-ad6bdf25128c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:26:55.256633 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e97b2-65c6-4235-972c-b386070c0709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:55.257455 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e97b2-65c6-4235-972c-b386070c0709 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:55.257765 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e97b2-65c6-4235-972c-b386070c0709 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:55.258658 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e97b2-65c6-4235-972c-b386070c0709 None admin] PATH_INFO: `/v3/projects/2ce34355c4f84047a2f4e1d1adcee45f` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:55.259958 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7e97b2-65c6-4235-972c-b386070c0709 None admin] RBAC: Authorizing `identity:get_project(project_id=2ce34355c4f84047a2f4e1d1adcee45f)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:55.261459 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7e97b2-65c6-4235-972c-b386070c0709 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:55.265056 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 71/137] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:26:54 2022] GET /identity/v3/projects/2ce34355c4f84047a2f4e1d1adcee45f => generated 312 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:26:55.271592 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e97b2-65c6-4235-972c-b386070c0709 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:55.304808 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:55.318097 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 67/138] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:26:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 08:26:55.374780 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f7af2e-86ba-4daa-865b-ad6bdf25128c None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:55.424139 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e97b2-65c6-4235-972c-b386070c0709 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:55.490078 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf419292-4d51-4b15-89f9-7a5512739ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:55.494441 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf419292-4d51-4b15-89f9-7a5512739ca0 None admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:55.495018 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf419292-4d51-4b15-89f9-7a5512739ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:55.495631 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf419292-4d51-4b15-89f9-7a5512739ca0 None admin] PATH_INFO: `/v3/projects/2ce34355c4f84047a2f4e1d1adcee45f/users/8e322e2cfd9e4eca9f58d562b4da2dad/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:55.497854 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:55.500854 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf419292-4d51-4b15-89f9-7a5512739ca0 None admin] RBAC: Authorizing `identity:create_grant(project_id=2ce34355c4f84047a2f4e1d1adcee45f, user_id=8e322e2cfd9e4eca9f58d562b4da2dad, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:55.504170 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf419292-4d51-4b15-89f9-7a5512739ca0 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:55.529345 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 72/139] 10.222.0.13 () {58 vars in 1439 bytes} [Fri Jun 24 08:26:55 2022] PUT /identity/v3/projects/2ce34355c4f84047a2f4e1d1adcee45f/users/8e322e2cfd9e4eca9f58d562b4da2dad/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:26:55.550199 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f7af2e-86ba-4daa-865b-ad6bdf25128c None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:55.597167 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a068f7f6-74bd-42ec-b9c6-3c86e65708f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:55.598076 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a068f7f6-74bd-42ec-b9c6-3c86e65708f7 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:55.598562 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a068f7f6-74bd-42ec-b9c6-3c86e65708f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:55.599021 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a068f7f6-74bd-42ec-b9c6-3c86e65708f7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:55.600786 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a068f7f6-74bd-42ec-b9c6-3c86e65708f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:55.603284 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a068f7f6-74bd-42ec-b9c6-3c86e65708f7 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:55.605512 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 68/140] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:26:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:26:55.611177 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf419292-4d51-4b15-89f9-7a5512739ca0 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:55.726445 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:55.758126 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf419292-4d51-4b15-89f9-7a5512739ca0 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:55.808818 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cfd318-481f-468c-b2bf-6bb6d8928dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:55.809422 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cfd318-481f-468c-b2bf-6bb6d8928dd1 None admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:55.809726 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cfd318-481f-468c-b2bf-6bb6d8928dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:55.810013 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cfd318-481f-468c-b2bf-6bb6d8928dd1 None admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:55.810734 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cfd318-481f-468c-b2bf-6bb6d8928dd1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:55.812193 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cfd318-481f-468c-b2bf-6bb6d8928dd1 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:55.852241 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 73/141] 10.222.0.13 () {60 vars in 1132 bytes} [Fri Jun 24 08:26:55 2022] POST /identity/v3/users => generated 283 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:26:56.216027 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc483a0f-fbb2-4400-8fa4-856d867ae114 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:56.216558 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc483a0f-fbb2-4400-8fa4-856d867ae114 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:56.216558 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc483a0f-fbb2-4400-8fa4-856d867ae114 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:56.217995 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 69/142] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:26:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:26:56.223370 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb13f3-1538-4796-8956-ef17c539b208 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:56.223667 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb13f3-1538-4796-8956-ef17c539b208 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:56.223952 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb13f3-1538-4796-8956-ef17c539b208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:56.241645 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-d5cb13f3-1538-4796-8956-ef17c539b208 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:26:56.243521 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-d5cb13f3-1538-4796-8956-ef17c539b208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:26:56.336046 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 74/143] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:26:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:26:56.376031 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc483a0f-fbb2-4400-8fa4-856d867ae114 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:56.510482 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc483a0f-fbb2-4400-8fa4-856d867ae114 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:56.548546 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca23b67-af70-427f-bdc2-c3b6924b503b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:56.549341 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca23b67-af70-427f-bdc2-c3b6924b503b None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:56.549627 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca23b67-af70-427f-bdc2-c3b6924b503b None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:56.549951 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca23b67-af70-427f-bdc2-c3b6924b503b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:56.552636 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:56.562737 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca23b67-af70-427f-bdc2-c3b6924b503b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:26:56.571285 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-2ca23b67-af70-427f-bdc2-c3b6924b503b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:26:56.571285 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:26:56.571285 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:26:56.571285 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:26:56.571285 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:26:56.571285 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:26:56.571285 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:26:56.571285 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:26:56.571285 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:26:56.571285 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:26:56.571285 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:26:56.571285 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:26:56.571285 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:26:56.571285 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:26:56.571285 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:26:56.571285 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:26:56.572528 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:26:56.572528 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:26:56.572528 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:26:56.572528 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:26:56.572528 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:26:56.572528 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:26:56.572528 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:26:56.572528 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:26:56.572528 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:26:56.572528 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:26:56.572528 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:26:56.572528 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:26:56.572528 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:26:56.572528 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:26:56.572528 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:26:56.572528 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:26:56.573362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:26:56.573362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:26:56.573362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:26:56.573362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:26:56.573362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:26:56.573362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:26:56.573362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:26:56.573362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:26:56.573362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:26:56.573362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:26:56.573362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:26:56.573362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:26:56.573362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 70/144] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:26:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:26:56.578490 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cb13f3-1538-4796-8956-ef17c539b208 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:56.663286 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:56.714218 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cb13f3-1538-4796-8956-ef17c539b208 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:56.756842 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f04fbe-c68c-440a-a8b9-b5c60e57ba5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:56.757606 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f04fbe-c68c-440a-a8b9-b5c60e57ba5e None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:56.757844 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f04fbe-c68c-440a-a8b9-b5c60e57ba5e None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:56.759139 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f04fbe-c68c-440a-a8b9-b5c60e57ba5e None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:56.759420 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f04fbe-c68c-440a-a8b9-b5c60e57ba5e None admin] RBAC: Adding query filter params (name=member) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:26:56.759731 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f04fbe-c68c-440a-a8b9-b5c60e57ba5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:56.761213 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f04fbe-c68c-440a-a8b9-b5c60e57ba5e None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:56.770217 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 75/145] 10.222.0.13 () {56 vars in 1101 bytes} [Fri Jun 24 08:26:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:26:56.778797 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca23b67-af70-427f-bdc2-c3b6924b503b None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:56.924420 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca23b67-af70-427f-bdc2-c3b6924b503b None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:56.971399 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66074bf-bd26-4f1e-8cc6-bfc67641797f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:56.972290 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66074bf-bd26-4f1e-8cc6-bfc67641797f None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:56.972574 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66074bf-bd26-4f1e-8cc6-bfc67641797f None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:56.972847 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66074bf-bd26-4f1e-8cc6-bfc67641797f None admin] PATH_INFO: `/v3/users/8e322e2cfd9e4eca9f58d562b4da2dad` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:56.974112 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66074bf-bd26-4f1e-8cc6-bfc67641797f None admin] RBAC: Authorizing `identity:get_user(user_id=8e322e2cfd9e4eca9f58d562b4da2dad)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:56.975583 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66074bf-bd26-4f1e-8cc6-bfc67641797f None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:56.977998 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 71/146] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Jun 24 08:26:56 2022] GET /identity/v3/users/8e322e2cfd9e4eca9f58d562b4da2dad => generated 275 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:26:56.984305 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f04fbe-c68c-440a-a8b9-b5c60e57ba5e None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:57.104463 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f04fbe-c68c-440a-a8b9-b5c60e57ba5e None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:57.136105 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:57.147190 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a1d3ae-f573-4a80-bf72-a3124a45715f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:57.148232 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a1d3ae-f573-4a80-bf72-a3124a45715f None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:57.149040 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a1d3ae-f573-4a80-bf72-a3124a45715f None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:57.149314 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a1d3ae-f573-4a80-bf72-a3124a45715f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:57.229570 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a1d3ae-f573-4a80-bf72-a3124a45715f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:57.231087 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a1d3ae-f573-4a80-bf72-a3124a45715f None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:57.285475 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ef0e9-6a04-47a1-881c-0ad473c7469e None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:57.286084 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ef0e9-6a04-47a1-881c-0ad473c7469e None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:57.286084 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ef0e9-6a04-47a1-881c-0ad473c7469e None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:57.287690 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 72/147] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:26:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:26:57.294924 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb071a-5988-4257-85e8-574ac8ae719e None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:57.295274 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb071a-5988-4257-85e8-574ac8ae719e None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:57.295556 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb071a-5988-4257-85e8-574ac8ae719e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:57.318070 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-ffdb071a-5988-4257-85e8-574ac8ae719e None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:26:57.319305 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-ffdb071a-5988-4257-85e8-574ac8ae719e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:26:57.405540 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 76/148] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:26:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:26:57.411488 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a1d3ae-f573-4a80-bf72-a3124a45715f None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:57.459153 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:57.474783 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:57.502536 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 73/149] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:26:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:26:57.550813 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb071a-5988-4257-85e8-574ac8ae719e None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:57.561941 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a1d3ae-f573-4a80-bf72-a3124a45715f None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:57.609309 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1f53da-de89-498b-b773-a89d9a7f3df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:57.610633 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1f53da-de89-498b-b773-a89d9a7f3df6 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:57.611058 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1f53da-de89-498b-b773-a89d9a7f3df6 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:57.611490 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1f53da-de89-498b-b773-a89d9a7f3df6 None admin] PATH_INFO: `/v3/projects/0a1cff337cd248ba94724654a9527e19` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:57.612860 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1f53da-de89-498b-b773-a89d9a7f3df6 None admin] RBAC: Authorizing `identity:get_project(project_id=0a1cff337cd248ba94724654a9527e19)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:57.615369 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1f53da-de89-498b-b773-a89d9a7f3df6 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:57.617527 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 77/150] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:26:57 2022] GET /identity/v3/projects/0a1cff337cd248ba94724654a9527e19 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:26:57.624270 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1f53da-de89-498b-b773-a89d9a7f3df6 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:57.749769 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1f53da-de89-498b-b773-a89d9a7f3df6 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:57.774903 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:57.798946 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb071a-5988-4257-85e8-574ac8ae719e None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:57.808270 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2c1830-835b-4445-b4bd-4cedac512f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:57.809236 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2c1830-835b-4445-b4bd-4cedac512f91 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:57.809758 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2c1830-835b-4445-b4bd-4cedac512f91 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:57.810202 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2c1830-835b-4445-b4bd-4cedac512f91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:57.811231 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2c1830-835b-4445-b4bd-4cedac512f91 None admin] RBAC: Adding query filter params (role.id=bd56838b71ec4a5781fe2df9f3f4f1ab, scope.project.id=0a1cff337cd248ba94724654a9527e19, user.id=8e322e2cfd9e4eca9f58d562b4da2dad) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:26:57.811730 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2c1830-835b-4445-b4bd-4cedac512f91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:57.817960 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2c1830-835b-4445-b4bd-4cedac512f91 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:57.828865 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 78/151] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Jun 24 08:26:57 2022] GET /identity/v3/role_assignments?user.id=8e322e2cfd9e4eca9f58d562b4da2dad&scope.project.id=0a1cff337cd248ba94724654a9527e19&role.id=bd56838b71ec4a5781fe2df9f3f4f1ab => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:26:57.869019 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de839017-300d-4e4d-821d-f75522c2303f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:57.869875 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de839017-300d-4e4d-821d-f75522c2303f None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:57.870183 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de839017-300d-4e4d-821d-f75522c2303f None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:57.870467 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de839017-300d-4e4d-821d-f75522c2303f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:57.894606 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de839017-300d-4e4d-821d-f75522c2303f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:26:57.902292 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-de839017-300d-4e4d-821d-f75522c2303f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 08:26:57.902292 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:26:57.902292 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:26:57.902292 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:26:57.902292 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:26:57.902292 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:26:57.902292 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:26:57.902292 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:26:57.902292 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:26:57.902292 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:26:57.902292 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:26:57.902292 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:26:57.902292 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:26:57.902292 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:26:57.902292 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:26:57.902292 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:26:57.903104 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:26:57.903104 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:26:57.903104 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:26:57.903104 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:26:57.903104 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:26:57.903104 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:26:57.903104 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:26:57.903104 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:26:57.903104 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:26:57.903104 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:26:57.903104 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:26:57.903104 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:26:57.903104 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:26:57.903104 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:26:57.903104 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:26:57.903104 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:26:57.903841 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:26:57.903841 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:26:57.903841 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:26:57.903841 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:26:57.903841 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:26:57.903841 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:26:57.903841 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:26:57.903841 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:26:57.903841 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:26:57.903841 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:26:57.903841 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 08:26:57.903841 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:26:57.903841 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 74/152] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:26:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:26:57.911256 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2c1830-835b-4445-b4bd-4cedac512f91 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:58.110313 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2c1830-835b-4445-b4bd-4cedac512f91 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:58.158489 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8695b617-ddf0-4d6c-8274-56395b046d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:58.159102 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695b617-ddf0-4d6c-8274-56395b046d2c None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:58.159384 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695b617-ddf0-4d6c-8274-56395b046d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:58.159658 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695b617-ddf0-4d6c-8274-56395b046d2c None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:58.160419 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8695b617-ddf0-4d6c-8274-56395b046d2c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:26:58.160725 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8695b617-ddf0-4d6c-8274-56395b046d2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:58.162138 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8695b617-ddf0-4d6c-8274-56395b046d2c None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:58.163341 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb05fdc2-022c-4d2f-a0ab-0e0549e4ace7 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:58.163631 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb05fdc2-022c-4d2f-a0ab-0e0549e4ace7 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:58.163910 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb05fdc2-022c-4d2f-a0ab-0e0549e4ace7 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:58.170186 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 75/153] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:26:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:26:58.172593 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 79/154] 10.222.0.13 () {56 vars in 1101 bytes} [Fri Jun 24 08:26:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:26:58.176344 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed980ef0-f4d1-44f0-ab14-2baab7219d0a None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:58.176635 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed980ef0-f4d1-44f0-ab14-2baab7219d0a None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:58.176918 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed980ef0-f4d1-44f0-ab14-2baab7219d0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:58.178807 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8695b617-ddf0-4d6c-8274-56395b046d2c None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:58.206988 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-ed980ef0-f4d1-44f0-ab14-2baab7219d0a None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:26:58.208179 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-ed980ef0-f4d1-44f0-ab14-2baab7219d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:26:58.212369 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:58.316935 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8695b617-ddf0-4d6c-8274-56395b046d2c None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:58.344781 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:58.359611 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d9591-92a4-4821-b0e6-a19018295dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:58.365715 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d9591-92a4-4821-b0e6-a19018295dd8 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:58.365715 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d9591-92a4-4821-b0e6-a19018295dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:58.365715 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d9591-92a4-4821-b0e6-a19018295dd8 None admin] PATH_INFO: `/v3/users/647a5533a0ab41cab27f26ff9d07ff5b` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:58.365715 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060d9591-92a4-4821-b0e6-a19018295dd8 None admin] RBAC: Authorizing `identity:get_user(user_id=647a5533a0ab41cab27f26ff9d07ff5b)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:58.365715 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060d9591-92a4-4821-b0e6-a19018295dd8 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:58.366460 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 80/155] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Jun 24 08:26:58 2022] GET /identity/v3/users/647a5533a0ab41cab27f26ff9d07ff5b => generated 289 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:26:58.378806 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d9591-92a4-4821-b0e6-a19018295dd8 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:58.404408 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 76/156] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:26:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:26:58.447953 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed980ef0-f4d1-44f0-ab14-2baab7219d0a None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:58.555410 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:58.597619 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d9591-92a4-4821-b0e6-a19018295dd8 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:58.662471 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed980ef0-f4d1-44f0-ab14-2baab7219d0a None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:58.685164 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd5bc14-d770-4a6d-9794-1b2e13790cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:58.686070 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd5bc14-d770-4a6d-9794-1b2e13790cdb None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:58.686355 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd5bc14-d770-4a6d-9794-1b2e13790cdb None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:58.686633 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd5bc14-d770-4a6d-9794-1b2e13790cdb None admin] PATH_INFO: `/v3/projects/0a1cff337cd248ba94724654a9527e19` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:58.687848 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd5bc14-d770-4a6d-9794-1b2e13790cdb None admin] RBAC: Authorizing `identity:get_project(project_id=0a1cff337cd248ba94724654a9527e19)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:58.689302 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd5bc14-d770-4a6d-9794-1b2e13790cdb None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:58.691057 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 81/157] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:26:58 2022] GET /identity/v3/projects/0a1cff337cd248ba94724654a9527e19 => generated 298 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:26:58.697479 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd5bc14-d770-4a6d-9794-1b2e13790cdb None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:58.743090 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c3dffd-a7d4-48f7-a0ce-16f22d3077b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:58.743926 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3dffd-a7d4-48f7-a0ce-16f22d3077b1 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:58.744320 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3dffd-a7d4-48f7-a0ce-16f22d3077b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:58.744597 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3dffd-a7d4-48f7-a0ce-16f22d3077b1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:58.775324 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c3dffd-a7d4-48f7-a0ce-16f22d3077b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:26:58.783307 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-f0c3dffd-a7d4-48f7-a0ce-16f22d3077b1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 08:26:58.783307 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:26:58.783307 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:26:58.783307 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:26:58.783307 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:26:58.783307 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:26:58.783307 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:26:58.783307 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:26:58.783307 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:26:58.783307 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:26:58.783307 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:26:58.783307 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:26:58.783307 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:26:58.783307 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:26:58.783307 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:26:58.783307 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:26:58.784255 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:26:58.784255 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:26:58.784255 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:26:58.784255 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:26:58.784255 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:26:58.784255 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:26:58.784255 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:26:58.784255 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:26:58.784255 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:26:58.784255 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:26:58.784255 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:26:58.784255 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:26:58.784255 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:26:58.784255 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:26:58.784255 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:26:58.784255 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:26:58.785003 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:26:58.785003 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:26:58.785003 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:26:58.785003 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:26:58.785003 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:26:58.785003 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:26:58.785003 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:26:58.785003 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:26:58.785003 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:26:58.785003 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:26:58.785003 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 08:26:58.785003 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:26:58.785003 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 77/158] 10.222.0.13 () {56 vars in 1114 bytes} [Fri Jun 24 08:26:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:26:58.813467 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c3dffd-a7d4-48f7-a0ce-16f22d3077b1 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:58.861202 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd5bc14-d770-4a6d-9794-1b2e13790cdb None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:58.913759 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f078a783-55c3-4601-aa28-3fd7569bb80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:58.915882 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f078a783-55c3-4601-aa28-3fd7569bb80a None admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:58.916181 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f078a783-55c3-4601-aa28-3fd7569bb80a None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:58.916463 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f078a783-55c3-4601-aa28-3fd7569bb80a None admin] PATH_INFO: `/v3/projects/0a1cff337cd248ba94724654a9527e19/users/647a5533a0ab41cab27f26ff9d07ff5b/roles/986a50f3abf14fdab55012a8e52719c7` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:58.920575 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:58.921612 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f078a783-55c3-4601-aa28-3fd7569bb80a None admin] RBAC: Authorizing `identity:create_grant(project_id=0a1cff337cd248ba94724654a9527e19, user_id=647a5533a0ab41cab27f26ff9d07ff5b, role_id=986a50f3abf14fdab55012a8e52719c7)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:58.923631 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f078a783-55c3-4601-aa28-3fd7569bb80a None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:58.958363 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 82/159] 10.222.0.13 () {58 vars in 1439 bytes} [Fri Jun 24 08:26:58 2022] PUT /identity/v3/projects/0a1cff337cd248ba94724654a9527e19/users/647a5533a0ab41cab27f26ff9d07ff5b/roles/986a50f3abf14fdab55012a8e52719c7 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:26:58.960660 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e2c9fd-0863-40a6-911f-88819e3bebda None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:58.960957 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e2c9fd-0863-40a6-911f-88819e3bebda None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:58.961270 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e2c9fd-0863-40a6-911f-88819e3bebda None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:58.962679 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 83/160] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:26:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:26:58.968472 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5cb766-0adb-4723-80b5-5d90d18d63d2 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:58.968750 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5cb766-0adb-4723-80b5-5d90d18d63d2 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:58.969029 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5cb766-0adb-4723-80b5-5d90d18d63d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:58.988696 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-9b5cb766-0adb-4723-80b5-5d90d18d63d2 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:26:58.990016 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-9b5cb766-0adb-4723-80b5-5d90d18d63d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:26:59.008953 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:59.064171 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c3dffd-a7d4-48f7-a0ce-16f22d3077b1 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:59.150449 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e103ff98-b01c-4de8-9b35-f67e2ea97fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:59.151138 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103ff98-b01c-4de8-9b35-f67e2ea97fd4 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:59.151421 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103ff98-b01c-4de8-9b35-f67e2ea97fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:59.151698 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103ff98-b01c-4de8-9b35-f67e2ea97fd4 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:59.152467 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e103ff98-b01c-4de8-9b35-f67e2ea97fd4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:26:59.152773 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e103ff98-b01c-4de8-9b35-f67e2ea97fd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:59.154241 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e103ff98-b01c-4de8-9b35-f67e2ea97fd4 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:59.157340 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 84/161] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:26:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:26:59.173040 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 78/162] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:26:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:26:59.178363 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5cb766-0adb-4723-80b5-5d90d18d63d2 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:59.202349 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e103ff98-b01c-4de8-9b35-f67e2ea97fd4 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:59.368620 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5cb766-0adb-4723-80b5-5d90d18d63d2 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:59.413402 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e103ff98-b01c-4de8-9b35-f67e2ea97fd4 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:59.423327 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:59.480832 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a98cbea-9c06-4eba-a9de-e8f3817462a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:59.481745 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98cbea-9c06-4eba-a9de-e8f3817462a4 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:59.482035 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98cbea-9c06-4eba-a9de-e8f3817462a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:59.482303 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98cbea-9c06-4eba-a9de-e8f3817462a4 None admin] PATH_INFO: `/v3/users/8e322e2cfd9e4eca9f58d562b4da2dad` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:59.483611 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a98cbea-9c06-4eba-a9de-e8f3817462a4 None admin] RBAC: Authorizing `identity:get_user(user_id=8e322e2cfd9e4eca9f58d562b4da2dad)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:59.485128 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a98cbea-9c06-4eba-a9de-e8f3817462a4 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:59.486940 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 85/163] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Jun 24 08:26:59 2022] GET /identity/v3/users/8e322e2cfd9e4eca9f58d562b4da2dad => generated 275 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:26:59.493162 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a98cbea-9c06-4eba-a9de-e8f3817462a4 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:59.498386 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8893c43d-28e7-4635-8420-2df9915d3fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:59.499743 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893c43d-28e7-4635-8420-2df9915d3fad None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:59.500218 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893c43d-28e7-4635-8420-2df9915d3fad None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:59.500692 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893c43d-28e7-4635-8420-2df9915d3fad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:59.519229 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:26:59.526608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8893c43d-28e7-4635-8420-2df9915d3fad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:26:59.534409 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-8893c43d-28e7-4635-8420-2df9915d3fad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:26:59.534409 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:26:59.534409 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:26:59.534409 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:26:59.534409 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:26:59.534409 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:26:59.534409 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:26:59.534409 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:26:59.534409 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:26:59.534409 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:26:59.534409 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:26:59.534409 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:26:59.534409 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:26:59.534409 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:26:59.534409 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:26:59.534409 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:26:59.535262 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:26:59.535262 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:26:59.535262 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:26:59.535262 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:26:59.535262 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:26:59.535262 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:26:59.535262 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:26:59.535262 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:26:59.535262 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:26:59.535262 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:26:59.535262 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:26:59.535262 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:26:59.535262 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:26:59.535262 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:26:59.535262 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:26:59.535262 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:26:59.536025 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:26:59.536025 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:26:59.536025 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:26:59.536025 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:26:59.536025 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:26:59.536025 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:26:59.536025 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:26:59.536025 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:26:59.536025 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:26:59.536025 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:26:59.536025 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:26:59.536025 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:26:59.536025 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 79/164] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Jun 24 08:26:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:26:59.542880 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8893c43d-28e7-4635-8420-2df9915d3fad None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:59.760449 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a98cbea-9c06-4eba-a9de-e8f3817462a4 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:59.806313 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8893c43d-28e7-4635-8420-2df9915d3fad None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:26:59.835436 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da41ff6-ea27-4009-9500-5340d4b7bbc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:59.836170 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da41ff6-ea27-4009-9500-5340d4b7bbc4 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:59.836478 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da41ff6-ea27-4009-9500-5340d4b7bbc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:59.837122 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da41ff6-ea27-4009-9500-5340d4b7bbc4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:59.874256 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c056d0-485b-4fd8-9315-cbf8fe1f6814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:26:59.874978 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c056d0-485b-4fd8-9315-cbf8fe1f6814 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:59.875267 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c056d0-485b-4fd8-9315-cbf8fe1f6814 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:59.875545 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c056d0-485b-4fd8-9315-cbf8fe1f6814 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:59.876288 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c056d0-485b-4fd8-9315-cbf8fe1f6814 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:26:59.876585 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c056d0-485b-4fd8-9315-cbf8fe1f6814 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:26:59.878043 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c056d0-485b-4fd8-9315-cbf8fe1f6814 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:26:59.893067 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 80/165] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:26:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:26:59.894374 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d28248-7bcc-4377-939a-8b4334d1af98 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:26:59.894666 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d28248-7bcc-4377-939a-8b4334d1af98 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:26:59.895016 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d28248-7bcc-4377-939a-8b4334d1af98 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:26:59.896516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 81/166] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:26:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:26:59.902335 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d28248-7bcc-4377-939a-8b4334d1af98 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:26:59.938068 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:00.006628 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da41ff6-ea27-4009-9500-5340d4b7bbc4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:00.008336 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da41ff6-ea27-4009-9500-5340d4b7bbc4 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:00.040356 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:00.168970 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d28248-7bcc-4377-939a-8b4334d1af98 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:00.206803 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 86/167] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:26:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 714 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 08:27:00.208390 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e65d5b3-f910-4fc6-806f-4a541b6c134e None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:00.208859 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e65d5b3-f910-4fc6-806f-4a541b6c134e None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:00.209312 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e65d5b3-f910-4fc6-806f-4a541b6c134e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:00.234416 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0165fcc5-356b-43d0-84e3-3afa6fb3e5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:00.236508 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0165fcc5-356b-43d0-84e3-3afa6fb3e5ae None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:00.236508 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0165fcc5-356b-43d0-84e3-3afa6fb3e5ae None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:00.236508 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0165fcc5-356b-43d0-84e3-3afa6fb3e5ae None admin] PATH_INFO: `/v3/users/268e75b173ff435b8dfbe2517f69c015` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:00.237498 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0165fcc5-356b-43d0-84e3-3afa6fb3e5ae None admin] RBAC: Authorizing `identity:get_user(user_id=268e75b173ff435b8dfbe2517f69c015)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:00.238069 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-3e65d5b3-f910-4fc6-806f-4a541b6c134e None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:00.238994 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0165fcc5-356b-43d0-84e3-3afa6fb3e5ae None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:00.239669 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-3e65d5b3-f910-4fc6-806f-4a541b6c134e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:00.240641 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 82/168] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Jun 24 08:26:59 2022] GET /identity/v3/users/268e75b173ff435b8dfbe2517f69c015 => generated 247 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:00.245715 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0165fcc5-356b-43d0-84e3-3afa6fb3e5ae None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:00.372344 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 87/169] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:27:00.373554 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e65d5b3-f910-4fc6-806f-4a541b6c134e None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:00.377711 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:00.381553 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0165fcc5-356b-43d0-84e3-3afa6fb3e5ae None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:00.417671 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd7f574-75c0-4893-8a37-92b3f7080013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:00.418903 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd7f574-75c0-4893-8a37-92b3f7080013 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:00.419188 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd7f574-75c0-4893-8a37-92b3f7080013 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:00.419462 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd7f574-75c0-4893-8a37-92b3f7080013 None admin] PATH_INFO: `/v3/projects/0a1cff337cd248ba94724654a9527e19` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:00.420684 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd7f574-75c0-4893-8a37-92b3f7080013 None admin] RBAC: Authorizing `identity:get_project(project_id=0a1cff337cd248ba94724654a9527e19)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:00.422268 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd7f574-75c0-4893-8a37-92b3f7080013 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:00.424277 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 83/170] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:27:00 2022] GET /identity/v3/projects/0a1cff337cd248ba94724654a9527e19 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:00.430797 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd7f574-75c0-4893-8a37-92b3f7080013 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:00.460674 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:00.539028 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd7f574-75c0-4893-8a37-92b3f7080013 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:00.575533 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbed680-e342-463a-81d1-11e0a124a791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:00.575733 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e65d5b3-f910-4fc6-806f-4a541b6c134e None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:00.576191 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbed680-e342-463a-81d1-11e0a124a791 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:00.576478 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbed680-e342-463a-81d1-11e0a124a791 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:00.576756 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbed680-e342-463a-81d1-11e0a124a791 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:00.577596 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbed680-e342-463a-81d1-11e0a124a791 None admin] RBAC: Adding query filter params (role.id=d6ada62a74a94222b864a6c283f8d9b1, scope.project.id=0a1cff337cd248ba94724654a9527e19, user.id=8e322e2cfd9e4eca9f58d562b4da2dad) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:00.577936 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbed680-e342-463a-81d1-11e0a124a791 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:00.579611 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbed680-e342-463a-81d1-11e0a124a791 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:00.589834 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 84/171] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Jun 24 08:27:00 2022] GET /identity/v3/role_assignments?user.id=8e322e2cfd9e4eca9f58d562b4da2dad&scope.project.id=0a1cff337cd248ba94724654a9527e19&role.id=d6ada62a74a94222b864a6c283f8d9b1 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:00.590872 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbed680-e342-463a-81d1-11e0a124a791 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:00.636043 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:00.659600 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4275a3c6-4b96-49fc-9ecc-163d5885f5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:00.660272 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275a3c6-4b96-49fc-9ecc-163d5885f5a3 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:00.660663 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275a3c6-4b96-49fc-9ecc-163d5885f5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:00.660952 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275a3c6-4b96-49fc-9ecc-163d5885f5a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:00.732874 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbed680-e342-463a-81d1-11e0a124a791 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:00.770773 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1402c4f-bb25-4670-852f-7b6d61a14987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:00.771579 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1402c4f-bb25-4670-852f-7b6d61a14987 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:00.771863 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1402c4f-bb25-4670-852f-7b6d61a14987 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:00.772141 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1402c4f-bb25-4670-852f-7b6d61a14987 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:00.783921 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1402c4f-bb25-4670-852f-7b6d61a14987 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:27:00.792424 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-b1402c4f-bb25-4670-852f-7b6d61a14987 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:27:00.792424 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:00.792424 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:00.792424 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:00.792424 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:00.792424 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:00.792424 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:00.792424 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:00.792424 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:00.792424 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:00.792424 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:00.792424 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:00.792424 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:27:00.792424 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:27:00.792424 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:27:00.792424 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:27:00.793422 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:27:00.793422 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:27:00.793422 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:27:00.793422 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:27:00.793422 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:00.793422 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:00.793422 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:00.793422 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:00.793422 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:00.793422 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:00.793422 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:00.793422 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:00.793422 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:00.793422 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:00.793422 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:00.793422 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:00.794238 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:00.794238 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:00.794238 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:00.794238 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:00.794238 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:27:00.794238 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:27:00.794238 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:27:00.794238 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:27:00.794238 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:27:00.794238 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:27:00.794238 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:27:00.794238 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:27:00.794238 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 85/172] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:27:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:00.794863 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4275a3c6-4b96-49fc-9ecc-163d5885f5a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:00.796281 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4275a3c6-4b96-49fc-9ecc-163d5885f5a3 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:00.803536 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1402c4f-bb25-4670-852f-7b6d61a14987 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:00.906191 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:00.907846 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:00.918873 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1402c4f-bb25-4670-852f-7b6d61a14987 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:00.956384 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f71589-9abc-490b-986a-8c29569dfae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:00.957448 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f71589-9abc-490b-986a-8c29569dfae7 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:00.957730 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f71589-9abc-490b-986a-8c29569dfae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:00.958026 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f71589-9abc-490b-986a-8c29569dfae7 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:00.958783 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f71589-9abc-490b-986a-8c29569dfae7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:00.959091 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f71589-9abc-490b-986a-8c29569dfae7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:00.960495 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f71589-9abc-490b-986a-8c29569dfae7 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:00.971374 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 86/173] 10.222.0.13 () {56 vars in 1101 bytes} [Fri Jun 24 08:27:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:00.977830 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f78d158-2def-46ec-a1b2-b513f18a85ee None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:00.978329 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f78d158-2def-46ec-a1b2-b513f18a85ee None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:00.978615 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f78d158-2def-46ec-a1b2-b513f18a85ee None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:00.980376 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 87/174] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:00 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:00.981539 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f78d158-2def-46ec-a1b2-b513f18a85ee None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:01.044263 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 88/175] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:27:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 671 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:27:01.045365 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780da93e-2452-4130-866e-65843ede50d1 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:01.045660 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780da93e-2452-4130-866e-65843ede50d1 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:01.045982 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780da93e-2452-4130-866e-65843ede50d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:01.065348 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-780da93e-2452-4130-866e-65843ede50d1 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:01.066589 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-780da93e-2452-4130-866e-65843ede50d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:01.115445 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f78d158-2def-46ec-a1b2-b513f18a85ee None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:01.155340 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 89/176] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:27:01.158564 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7abac2-f62d-4ad4-923d-4a809c6d71c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:01.158987 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2232d4b-20d1-42f3-8ebf-0a65232811a6 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:01.159382 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7abac2-f62d-4ad4-923d-4a809c6d71c1 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:01.159666 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7abac2-f62d-4ad4-923d-4a809c6d71c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:01.159893 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2232d4b-20d1-42f3-8ebf-0a65232811a6 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:01.159989 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7abac2-f62d-4ad4-923d-4a809c6d71c1 None admin] PATH_INFO: `/v3/users/8e322e2cfd9e4eca9f58d562b4da2dad` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:01.160436 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2232d4b-20d1-42f3-8ebf-0a65232811a6 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:01.161133 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7abac2-f62d-4ad4-923d-4a809c6d71c1 None admin] RBAC: Authorizing `identity:get_user(user_id=8e322e2cfd9e4eca9f58d562b4da2dad)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:01.162205 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 90/177] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:01.162694 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7abac2-f62d-4ad4-923d-4a809c6d71c1 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:01.163364 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2232d4b-20d1-42f3-8ebf-0a65232811a6 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:01.164977 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 88/178] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Jun 24 08:27:00 2022] GET /identity/v3/users/8e322e2cfd9e4eca9f58d562b4da2dad => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:27:01.171931 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87351ec4-4810-4154-a2c2-776883e2940c None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:01.172223 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87351ec4-4810-4154-a2c2-776883e2940c None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:01.172507 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87351ec4-4810-4154-a2c2-776883e2940c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:01.194682 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-87351ec4-4810-4154-a2c2-776883e2940c None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:01.196725 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-87351ec4-4810-4154-a2c2-776883e2940c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:01.216215 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:01.261404 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:01.293264 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 89/179] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:27:01.294192 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87351ec4-4810-4154-a2c2-776883e2940c None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:01.304404 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2232d4b-20d1-42f3-8ebf-0a65232811a6 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:01.346886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfe17c4-5844-4518-a035-725a26bdb749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:01.347756 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe17c4-5844-4518-a035-725a26bdb749 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:01.348038 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe17c4-5844-4518-a035-725a26bdb749 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:01.348520 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe17c4-5844-4518-a035-725a26bdb749 None admin] PATH_INFO: `/v3/projects/0a1cff337cd248ba94724654a9527e19` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:01.349535 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfe17c4-5844-4518-a035-725a26bdb749 None admin] RBAC: Authorizing `identity:get_project(project_id=0a1cff337cd248ba94724654a9527e19)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:01.351034 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfe17c4-5844-4518-a035-725a26bdb749 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:01.353444 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 91/180] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:27:01 2022] GET /identity/v3/projects/0a1cff337cd248ba94724654a9527e19 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:01.358461 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfe17c4-5844-4518-a035-725a26bdb749 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:01.431570 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87351ec4-4810-4154-a2c2-776883e2940c None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:01.470860 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbb5184-e1f4-4728-b650-8f4f74b2bbbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:01.471508 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbb5184-e1f4-4728-b650-8f4f74b2bbbe None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:01.471793 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbb5184-e1f4-4728-b650-8f4f74b2bbbe None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:01.472081 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbb5184-e1f4-4728-b650-8f4f74b2bbbe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:01.475244 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfe17c4-5844-4518-a035-725a26bdb749 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:01.492529 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:01.512051 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465d662e-ecd6-4e74-b497-0cdc3c5d8e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:01.513192 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d662e-ecd6-4e74-b497-0cdc3c5d8e91 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:01.513192 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d662e-ecd6-4e74-b497-0cdc3c5d8e91 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:01.513421 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d662e-ecd6-4e74-b497-0cdc3c5d8e91 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:01.521610 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:01.526197 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465d662e-ecd6-4e74-b497-0cdc3c5d8e91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:27:01.534061 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-465d662e-ecd6-4e74-b497-0cdc3c5d8e91 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:27:01.534061 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:01.534061 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:01.534061 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:01.534061 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:01.534061 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:01.534061 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:01.534061 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:01.534061 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:01.534061 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:01.534061 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:01.534061 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:01.534061 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:27:01.534061 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:27:01.534061 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:27:01.534061 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:27:01.534936 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:27:01.534936 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:27:01.534936 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:27:01.534936 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:27:01.534936 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:01.534936 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:01.534936 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:01.534936 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:01.534936 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:01.534936 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:01.534936 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:01.534936 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:01.534936 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:01.534936 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:01.534936 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:01.534936 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:01.535727 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:01.535727 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:01.535727 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:01.535727 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:01.535727 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:27:01.535727 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:27:01.535727 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:27:01.535727 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:27:01.535727 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:27:01.535727 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:27:01.535727 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:27:01.535727 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:01.535727 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 92/181] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Jun 24 08:27:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:01.537070 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700356c1-c07a-40b6-8fdb-1a2b4e38b712 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:01.537070 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700356c1-c07a-40b6-8fdb-1a2b4e38b712 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:01.537070 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700356c1-c07a-40b6-8fdb-1a2b4e38b712 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:01.538439 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 93/182] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:01.540979 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700356c1-c07a-40b6-8fdb-1a2b4e38b712 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:01.557121 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbb5184-e1f4-4728-b650-8f4f74b2bbbe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:01.558654 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbb5184-e1f4-4728-b650-8f4f74b2bbbe None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:01.656644 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700356c1-c07a-40b6-8fdb-1a2b4e38b712 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:01.666116 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 90/183] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:27:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:27:01.667112 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbb5184-e1f4-4728-b650-8f4f74b2bbbe None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:01.699082 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9d6014-10d2-4237-8764-9febf9e3b132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:01.699981 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d6014-10d2-4237-8764-9febf9e3b132 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:01.700590 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d6014-10d2-4237-8764-9febf9e3b132 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:01.700882 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d6014-10d2-4237-8764-9febf9e3b132 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:01.712190 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9d6014-10d2-4237-8764-9febf9e3b132 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:27:01.719524 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:01.721138 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-bd9d6014-10d2-4237-8764-9febf9e3b132 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 08:27:01.721138 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:01.721138 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:01.721138 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:01.721138 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:01.721138 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:01.721138 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:01.721138 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:01.721138 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:01.721138 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:01.721138 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:01.721138 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:01.721138 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:27:01.721138 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:27:01.721138 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:27:01.721138 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:27:01.722808 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:27:01.722808 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:27:01.722808 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:27:01.722808 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:27:01.722808 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:01.722808 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:01.722808 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:01.722808 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:01.722808 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:01.722808 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:01.722808 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:01.722808 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:01.722808 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:01.722808 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:01.722808 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:01.722808 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:01.724016 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:01.724016 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:01.724016 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:01.724016 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:01.724016 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:27:01.724016 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:27:01.724016 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:27:01.724016 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:27:01.724016 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:27:01.724016 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:27:01.724016 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 08:27:01.724016 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:01.724016 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 94/184] 10.222.0.13 () {56 vars in 1114 bytes} [Fri Jun 24 08:27:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:01.726556 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9d6014-10d2-4237-8764-9febf9e3b132 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:01.772703 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbb5184-e1f4-4728-b650-8f4f74b2bbbe None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:01.788727 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:01.807314 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ced4b-96bb-415a-bad5-4d43e4d9d44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:01.807964 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ced4b-96bb-415a-bad5-4d43e4d9d44a None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:01.808251 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ced4b-96bb-415a-bad5-4d43e4d9d44a None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:01.808529 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ced4b-96bb-415a-bad5-4d43e4d9d44a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:01.809353 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1ced4b-96bb-415a-bad5-4d43e4d9d44a None admin] RBAC: Adding query filter params (role.id=acfe707eee6944ad94d1915fc9ee7ea3, scope.project.id=0a1cff337cd248ba94724654a9527e19, user.id=268e75b173ff435b8dfbe2517f69c015) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:01.809669 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1ced4b-96bb-415a-bad5-4d43e4d9d44a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:01.811180 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1ced4b-96bb-415a-bad5-4d43e4d9d44a None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:01.817324 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 91/185] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Jun 24 08:27:01 2022] GET /identity/v3/role_assignments?user.id=268e75b173ff435b8dfbe2517f69c015&scope.project.id=0a1cff337cd248ba94724654a9527e19&role.id=acfe707eee6944ad94d1915fc9ee7ea3 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:01.818579 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af846d0b-1493-43ea-996b-15681466805e None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:01.818875 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af846d0b-1493-43ea-996b-15681466805e None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:01.819159 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af846d0b-1493-43ea-996b-15681466805e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:01.832184 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9d6014-10d2-4237-8764-9febf9e3b132 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:01.846248 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-af846d0b-1493-43ea-996b-15681466805e None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:01.847472 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-af846d0b-1493-43ea-996b-15681466805e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:01.877202 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efb5589-efb7-483c-ac24-2df24f091ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:01.878489 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb5589-efb7-483c-ac24-2df24f091ca5 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:01.879001 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb5589-efb7-483c-ac24-2df24f091ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:01.879416 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb5589-efb7-483c-ac24-2df24f091ca5 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:01.880416 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efb5589-efb7-483c-ac24-2df24f091ca5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:01.880760 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efb5589-efb7-483c-ac24-2df24f091ca5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:01.883147 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efb5589-efb7-483c-ac24-2df24f091ca5 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:01.893200 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 95/186] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:27:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:01.898737 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efb5589-efb7-483c-ac24-2df24f091ca5 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:01.926870 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 92/187] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:27:01.928299 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53913a96-6d75-4146-b9c3-1d137fcd3f83 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:01.928587 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53913a96-6d75-4146-b9c3-1d137fcd3f83 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:01.928859 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53913a96-6d75-4146-b9c3-1d137fcd3f83 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:01.931007 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 93/188] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:01.935766 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53913a96-6d75-4146-b9c3-1d137fcd3f83 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:01.988683 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:02.035831 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efb5589-efb7-483c-ac24-2df24f091ca5 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:02.053917 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53913a96-6d75-4146-b9c3-1d137fcd3f83 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:02.061009 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:02.073804 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e31b47-8582-40bf-b4a7-f97a8682cc19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:02.074664 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e31b47-8582-40bf-b4a7-f97a8682cc19 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:02.074952 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e31b47-8582-40bf-b4a7-f97a8682cc19 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:02.075219 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e31b47-8582-40bf-b4a7-f97a8682cc19 None admin] PATH_INFO: `/v3/projects/2ce34355c4f84047a2f4e1d1adcee45f` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:02.076391 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e31b47-8582-40bf-b4a7-f97a8682cc19 None admin] RBAC: Authorizing `identity:get_project(project_id=2ce34355c4f84047a2f4e1d1adcee45f)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:02.077852 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e31b47-8582-40bf-b4a7-f97a8682cc19 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:02.079747 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 96/189] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:27:01 2022] GET /identity/v3/projects/2ce34355c4f84047a2f4e1d1adcee45f => generated 312 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:02.080297 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e31b47-8582-40bf-b4a7-f97a8682cc19 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:02.090733 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bc8b02-0fe6-41b1-9f5c-df315c5586a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:02.091397 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc8b02-0fe6-41b1-9f5c-df315c5586a0 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:02.091677 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc8b02-0fe6-41b1-9f5c-df315c5586a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:02.091948 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc8b02-0fe6-41b1-9f5c-df315c5586a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:02.092698 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bc8b02-0fe6-41b1-9f5c-df315c5586a0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:02.093000 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bc8b02-0fe6-41b1-9f5c-df315c5586a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:02.094401 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bc8b02-0fe6-41b1-9f5c-df315c5586a0 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:02.102483 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 94/190] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:27:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:02.103655 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29491fd8-5fcd-4b57-a7cc-9b4eda4c50f3 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:02.103937 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29491fd8-5fcd-4b57-a7cc-9b4eda4c50f3 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:02.104210 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29491fd8-5fcd-4b57-a7cc-9b4eda4c50f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:02.120044 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-29491fd8-5fcd-4b57-a7cc-9b4eda4c50f3 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:02.121213 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-29491fd8-5fcd-4b57-a7cc-9b4eda4c50f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:02.183971 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e31b47-8582-40bf-b4a7-f97a8682cc19 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:02.186470 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 95/191] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:27:02.189874 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29491fd8-5fcd-4b57-a7cc-9b4eda4c50f3 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:02.223633 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:02.227198 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a5ab2c-fd37-4326-ae39-48d09dd46ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:02.228078 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5ab2c-fd37-4326-ae39-48d09dd46ba1 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:02.228367 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5ab2c-fd37-4326-ae39-48d09dd46ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:02.228647 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5ab2c-fd37-4326-ae39-48d09dd46ba1 None admin] PATH_INFO: `/v3/users/394ee1b1c13f4d2989c71839d7642eb2` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:02.242879 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a5ab2c-fd37-4326-ae39-48d09dd46ba1 None admin] RBAC: Authorizing `identity:get_user(user_id=394ee1b1c13f4d2989c71839d7642eb2)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:02.244292 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a5ab2c-fd37-4326-ae39-48d09dd46ba1 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:02.246014 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 97/192] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Jun 24 08:27:02 2022] GET /identity/v3/users/394ee1b1c13f4d2989c71839d7642eb2 => generated 283 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:02.246014 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a5ab2c-fd37-4326-ae39-48d09dd46ba1 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:02.295798 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:02.296404 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29491fd8-5fcd-4b57-a7cc-9b4eda4c50f3 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:02.330792 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da64ece-75f0-4e16-bf21-dc588abfc8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:02.331583 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da64ece-75f0-4e16-bf21-dc588abfc8fb None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:02.331871 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da64ece-75f0-4e16-bf21-dc588abfc8fb None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:02.332145 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da64ece-75f0-4e16-bf21-dc588abfc8fb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:02.333308 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da64ece-75f0-4e16-bf21-dc588abfc8fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:02.334802 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da64ece-75f0-4e16-bf21-dc588abfc8fb None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:02.337015 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 96/193] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:27:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:02.337792 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da64ece-75f0-4e16-bf21-dc588abfc8fb None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:02.352891 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a5ab2c-fd37-4326-ae39-48d09dd46ba1 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:02.388897 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e9a764-c2b4-4833-9e2b-88bc6fb0d958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:02.389564 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e9a764-c2b4-4833-9e2b-88bc6fb0d958 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:02.389885 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e9a764-c2b4-4833-9e2b-88bc6fb0d958 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:02.390166 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e9a764-c2b4-4833-9e2b-88bc6fb0d958 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:02.391000 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e9a764-c2b4-4833-9e2b-88bc6fb0d958 None admin] RBAC: Adding query filter params (role.id=bd56838b71ec4a5781fe2df9f3f4f1ab, scope.project.id=2ce34355c4f84047a2f4e1d1adcee45f, user.id=8e322e2cfd9e4eca9f58d562b4da2dad) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:02.391313 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e9a764-c2b4-4833-9e2b-88bc6fb0d958 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:02.392761 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e9a764-c2b4-4833-9e2b-88bc6fb0d958 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:02.399596 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 98/194] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Jun 24 08:27:02 2022] GET /identity/v3/role_assignments?user.id=8e322e2cfd9e4eca9f58d562b4da2dad&scope.project.id=2ce34355c4f84047a2f4e1d1adcee45f&role.id=bd56838b71ec4a5781fe2df9f3f4f1ab => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:02.400679 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e9a764-c2b4-4833-9e2b-88bc6fb0d958 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:02.447460 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da64ece-75f0-4e16-bf21-dc588abfc8fb None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:02.449249 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:02.501894 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc87e7df-a9db-413b-a2f9-8cdc1b3ba4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:02.503318 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc87e7df-a9db-413b-a2f9-8cdc1b3ba4d2 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:02.503810 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc87e7df-a9db-413b-a2f9-8cdc1b3ba4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:02.504301 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc87e7df-a9db-413b-a2f9-8cdc1b3ba4d2 None admin] PATH_INFO: `/v3/users/394ee1b1c13f4d2989c71839d7642eb2` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:02.506357 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc87e7df-a9db-413b-a2f9-8cdc1b3ba4d2 None admin] RBAC: Authorizing `identity:get_user(user_id=394ee1b1c13f4d2989c71839d7642eb2)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:02.507873 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc87e7df-a9db-413b-a2f9-8cdc1b3ba4d2 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:02.509654 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 97/195] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Jun 24 08:27:02 2022] GET /identity/v3/users/394ee1b1c13f4d2989c71839d7642eb2 => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:02.514619 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc87e7df-a9db-413b-a2f9-8cdc1b3ba4d2 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:02.523001 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e9a764-c2b4-4833-9e2b-88bc6fb0d958 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:02.538438 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:02.568313 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b22b7-484a-4cd5-bd44-e2b4cc77d30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:02.571252 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b22b7-484a-4cd5-bd44-e2b4cc77d30f None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:02.571697 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b22b7-484a-4cd5-bd44-e2b4cc77d30f None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:02.572133 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b22b7-484a-4cd5-bd44-e2b4cc77d30f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:02.589891 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5b22b7-484a-4cd5-bd44-e2b4cc77d30f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:27:02.603962 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-ad5b22b7-484a-4cd5-bd44-e2b4cc77d30f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 08:27:02.603962 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:02.603962 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:02.603962 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:02.603962 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:02.603962 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:02.603962 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:02.603962 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:02.603962 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:02.603962 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:02.603962 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:02.603962 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:02.603962 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:27:02.603962 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:27:02.603962 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:27:02.603962 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:27:02.604846 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:27:02.604846 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:27:02.604846 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:27:02.604846 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:27:02.604846 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:02.604846 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:02.604846 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:02.604846 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:02.604846 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:02.604846 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:02.604846 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:02.604846 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:02.604846 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:02.604846 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:02.604846 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:02.604846 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:02.605610 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:02.605610 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:02.605610 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:02.605610 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:02.605610 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:27:02.605610 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:27:02.605610 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:27:02.605610 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:27:02.605610 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:27:02.605610 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:27:02.605610 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 08:27:02.605610 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:02.606214 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 99/196] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:27:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:02.607330 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b22b7-484a-4cd5-bd44-e2b4cc77d30f None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:02.685346 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc87e7df-a9db-413b-a2f9-8cdc1b3ba4d2 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:02.721920 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3730ddf4-7056-402f-a323-2ecf87d6cf34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:02.722924 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3730ddf4-7056-402f-a323-2ecf87d6cf34 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:02.723241 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b22b7-484a-4cd5-bd44-e2b4cc77d30f None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:02.723365 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3730ddf4-7056-402f-a323-2ecf87d6cf34 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:02.723805 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3730ddf4-7056-402f-a323-2ecf87d6cf34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:02.785239 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b6caa7-a5af-41b5-8791-e108e3fc64be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:02.786018 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6caa7-a5af-41b5-8791-e108e3fc64be None admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:02.786390 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6caa7-a5af-41b5-8791-e108e3fc64be None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:02.787003 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6caa7-a5af-41b5-8791-e108e3fc64be None admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:02.787838 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b6caa7-a5af-41b5-8791-e108e3fc64be None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:02.790245 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b6caa7-a5af-41b5-8791-e108e3fc64be None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:02.791672 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:02.833722 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 100/197] 10.222.0.13 () {60 vars in 1132 bytes} [Fri Jun 24 08:27:02 2022] POST /identity/v3/users => generated 297 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:27:02.833722 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b6caa7-a5af-41b5-8791-e108e3fc64be None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:02.834031 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3730ddf4-7056-402f-a323-2ecf87d6cf34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:02.834031 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3730ddf4-7056-402f-a323-2ecf87d6cf34 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:02.857077 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:02.973411 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b6caa7-a5af-41b5-8791-e108e3fc64be None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:02.988926 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 98/198] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:27:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:27:02.990248 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3730ddf4-7056-402f-a323-2ecf87d6cf34 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:03.032024 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1640307-7297-4eda-b61b-3aef60bd9deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:03.032681 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1640307-7297-4eda-b61b-3aef60bd9deb None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:03.032964 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1640307-7297-4eda-b61b-3aef60bd9deb None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:03.033239 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1640307-7297-4eda-b61b-3aef60bd9deb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:03.091911 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:03.130639 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3730ddf4-7056-402f-a323-2ecf87d6cf34 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:03.134746 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1640307-7297-4eda-b61b-3aef60bd9deb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:03.136255 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1640307-7297-4eda-b61b-3aef60bd9deb None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:03.175438 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:03.184525 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5ecd1b-75b9-4299-90e7-49832fd2a82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:03.185214 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ecd1b-75b9-4299-90e7-49832fd2a82d None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:03.185502 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ecd1b-75b9-4299-90e7-49832fd2a82d None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:03.186661 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ecd1b-75b9-4299-90e7-49832fd2a82d None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:03.186763 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5ecd1b-75b9-4299-90e7-49832fd2a82d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:03.187060 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5ecd1b-75b9-4299-90e7-49832fd2a82d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:03.188487 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5ecd1b-75b9-4299-90e7-49832fd2a82d None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:03.201049 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 99/199] 10.222.0.13 () {56 vars in 1101 bytes} [Fri Jun 24 08:27:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:03.204247 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5ecd1b-75b9-4299-90e7-49832fd2a82d None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:03.259379 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 101/200] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:27:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:27:03.260304 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1640307-7297-4eda-b61b-3aef60bd9deb None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:03.322466 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5ecd1b-75b9-4299-90e7-49832fd2a82d None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:03.365968 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:03.371881 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1640307-7297-4eda-b61b-3aef60bd9deb None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:03.377759 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2f8bff-f3c9-42e0-a001-f791bf766d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:03.378630 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f8bff-f3c9-42e0-a001-f791bf766d54 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:03.378912 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f8bff-f3c9-42e0-a001-f791bf766d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:03.379185 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f8bff-f3c9-42e0-a001-f791bf766d54 None admin] PATH_INFO: `/v3/projects/58dfc569119141c688e8c2f5c4faab17` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:03.380926 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2f8bff-f3c9-42e0-a001-f791bf766d54 None admin] RBAC: Authorizing `identity:get_project(project_id=58dfc569119141c688e8c2f5c4faab17)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:03.383023 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2f8bff-f3c9-42e0-a001-f791bf766d54 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:03.386317 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 100/201] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:27:03 2022] GET /identity/v3/projects/58dfc569119141c688e8c2f5c4faab17 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:03.387523 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2f8bff-f3c9-42e0-a001-f791bf766d54 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:03.402326 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:03.415075 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429c485c-6c8a-4e90-994b-b12a0293747e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:03.416318 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c485c-6c8a-4e90-994b-b12a0293747e None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:03.416603 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c485c-6c8a-4e90-994b-b12a0293747e None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:03.416882 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c485c-6c8a-4e90-994b-b12a0293747e None admin] PATH_INFO: `/v3/users/647a5533a0ab41cab27f26ff9d07ff5b` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:03.418180 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429c485c-6c8a-4e90-994b-b12a0293747e None admin] RBAC: Authorizing `identity:get_user(user_id=647a5533a0ab41cab27f26ff9d07ff5b)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:03.420522 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429c485c-6c8a-4e90-994b-b12a0293747e None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:03.423236 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 102/202] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Jun 24 08:27:03 2022] GET /identity/v3/users/647a5533a0ab41cab27f26ff9d07ff5b => generated 289 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:03.424377 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429c485c-6c8a-4e90-994b-b12a0293747e None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:03.510091 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2f8bff-f3c9-42e0-a001-f791bf766d54 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:03.549334 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4012799-9985-4f3e-9255-3dc9fe2eadc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:03.550758 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4012799-9985-4f3e-9255-3dc9fe2eadc1 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:03.551055 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4012799-9985-4f3e-9255-3dc9fe2eadc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:03.551336 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4012799-9985-4f3e-9255-3dc9fe2eadc1 None admin] PATH_INFO: `/v3/projects/58dfc569119141c688e8c2f5c4faab17` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:03.552793 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4012799-9985-4f3e-9255-3dc9fe2eadc1 None admin] RBAC: Authorizing `identity:get_project(project_id=58dfc569119141c688e8c2f5c4faab17)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:03.554313 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4012799-9985-4f3e-9255-3dc9fe2eadc1 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:03.561382 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 101/203] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:27:03 2022] GET /identity/v3/projects/58dfc569119141c688e8c2f5c4faab17 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:03.561382 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4012799-9985-4f3e-9255-3dc9fe2eadc1 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:03.571815 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429c485c-6c8a-4e90-994b-b12a0293747e None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:03.610755 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d877ee-b26d-47b8-9f8a-031c10f1ce7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:03.611878 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d877ee-b26d-47b8-9f8a-031c10f1ce7e None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:03.612391 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d877ee-b26d-47b8-9f8a-031c10f1ce7e None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:03.612920 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d877ee-b26d-47b8-9f8a-031c10f1ce7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:03.613985 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d877ee-b26d-47b8-9f8a-031c10f1ce7e None admin] RBAC: Adding query filter params (role.id=acfe707eee6944ad94d1915fc9ee7ea3, scope.project.id=58dfc569119141c688e8c2f5c4faab17, user.id=394ee1b1c13f4d2989c71839d7642eb2) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:03.614489 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d877ee-b26d-47b8-9f8a-031c10f1ce7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:03.616255 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d877ee-b26d-47b8-9f8a-031c10f1ce7e None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:03.623199 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 103/204] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Jun 24 08:27:03 2022] GET /identity/v3/role_assignments?user.id=394ee1b1c13f4d2989c71839d7642eb2&scope.project.id=58dfc569119141c688e8c2f5c4faab17&role.id=acfe707eee6944ad94d1915fc9ee7ea3 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:03.624829 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d877ee-b26d-47b8-9f8a-031c10f1ce7e None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:03.629999 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:03.699237 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:03.745349 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4012799-9985-4f3e-9255-3dc9fe2eadc1 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:03.749296 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d877ee-b26d-47b8-9f8a-031c10f1ce7e None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:03.789158 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43ed453-7247-4ff8-96ef-075ccf783fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:03.790303 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ed453-7247-4ff8-96ef-075ccf783fff None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:03.790812 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ed453-7247-4ff8-96ef-075ccf783fff None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:03.791279 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ed453-7247-4ff8-96ef-075ccf783fff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:03.792332 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43ed453-7247-4ff8-96ef-075ccf783fff None admin] RBAC: Adding query filter params (role.id=d6ada62a74a94222b864a6c283f8d9b1, scope.project.id=58dfc569119141c688e8c2f5c4faab17, user.id=394ee1b1c13f4d2989c71839d7642eb2) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:03.792855 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43ed453-7247-4ff8-96ef-075ccf783fff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:03.795429 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43ed453-7247-4ff8-96ef-075ccf783fff None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:03.802573 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 104/205] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Jun 24 08:27:03 2022] GET /identity/v3/role_assignments?user.id=394ee1b1c13f4d2989c71839d7642eb2&scope.project.id=58dfc569119141c688e8c2f5c4faab17&role.id=d6ada62a74a94222b864a6c283f8d9b1 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:03.810717 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9527946-738d-4194-8e07-076c7dbfd635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:03.811466 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9527946-738d-4194-8e07-076c7dbfd635 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:03.811819 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9527946-738d-4194-8e07-076c7dbfd635 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:03.812163 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9527946-738d-4194-8e07-076c7dbfd635 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:03.929567 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9527946-738d-4194-8e07-076c7dbfd635 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:03.931957 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9527946-738d-4194-8e07-076c7dbfd635 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:04.022135 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:04.057006 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 102/206] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:27:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:27:04.062785 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43ed453-7247-4ff8-96ef-075ccf783fff None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:04.194068 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43ed453-7247-4ff8-96ef-075ccf783fff None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:04.217985 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:04.234763 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf9d1e7-6a30-4533-a619-44bd36fc69fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:04.236075 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf9d1e7-6a30-4533-a619-44bd36fc69fb None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:04.236590 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf9d1e7-6a30-4533-a619-44bd36fc69fb None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:04.237078 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf9d1e7-6a30-4533-a619-44bd36fc69fb None admin] PATH_INFO: `/v3/projects/0a1cff337cd248ba94724654a9527e19` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:04.242838 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf9d1e7-6a30-4533-a619-44bd36fc69fb None admin] RBAC: Authorizing `identity:get_project(project_id=0a1cff337cd248ba94724654a9527e19)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:04.244895 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf9d1e7-6a30-4533-a619-44bd36fc69fb None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:04.247254 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 105/207] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:27:04 2022] GET /identity/v3/projects/0a1cff337cd248ba94724654a9527e19 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:04.253386 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9527946-738d-4194-8e07-076c7dbfd635 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:04.399470 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9527946-738d-4194-8e07-076c7dbfd635 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:04.448275 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bd0c2d-106d-4ceb-a3cd-720024d331a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:04.448855 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd0c2d-106d-4ceb-a3cd-720024d331a7 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:04.449137 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd0c2d-106d-4ceb-a3cd-720024d331a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:04.449416 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd0c2d-106d-4ceb-a3cd-720024d331a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:04.450283 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bd0c2d-106d-4ceb-a3cd-720024d331a7 None admin] RBAC: Adding query filter params (role.id=986a50f3abf14fdab55012a8e52719c7, scope.project.id=0a1cff337cd248ba94724654a9527e19, user.id=647a5533a0ab41cab27f26ff9d07ff5b) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:04.450596 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bd0c2d-106d-4ceb-a3cd-720024d331a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:04.452040 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bd0c2d-106d-4ceb-a3cd-720024d331a7 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:04.459530 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 103/208] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Jun 24 08:27:04 2022] GET /identity/v3/role_assignments?user.id=647a5533a0ab41cab27f26ff9d07ff5b&scope.project.id=0a1cff337cd248ba94724654a9527e19&role.id=986a50f3abf14fdab55012a8e52719c7 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:05.340289 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986efcad-a7de-4c02-86b9-0cb6198ae59e None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:05.340804 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986efcad-a7de-4c02-86b9-0cb6198ae59e None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:05.340878 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986efcad-a7de-4c02-86b9-0cb6198ae59e None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:05.342268 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 106/209] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:05.347714 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced8fecb-e6f9-4afe-9325-861d59fc50d5 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:05.348479 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced8fecb-e6f9-4afe-9325-861d59fc50d5 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:05.348936 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced8fecb-e6f9-4afe-9325-861d59fc50d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:05.357252 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e666668-9de6-4110-a7f1-1d85ef6188ad None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:05.357846 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e666668-9de6-4110-a7f1-1d85ef6188ad None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:05.358317 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e666668-9de6-4110-a7f1-1d85ef6188ad None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:05.360133 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 107/210] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:05.365489 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139021c8-a94c-4339-bb0d-442b262cf402 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:05.366017 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139021c8-a94c-4339-bb0d-442b262cf402 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:05.366470 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139021c8-a94c-4339-bb0d-442b262cf402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:05.375011 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-ced8fecb-e6f9-4afe-9325-861d59fc50d5 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:05.376236 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-ced8fecb-e6f9-4afe-9325-861d59fc50d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:05.385030 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-139021c8-a94c-4339-bb0d-442b262cf402 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:05.386489 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-139021c8-a94c-4339-bb0d-442b262cf402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:05.404109 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:05.457711 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 104/211] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 08:27:05.475713 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced8fecb-e6f9-4afe-9325-861d59fc50d5 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:05.479452 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 108/212] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 08:27:05.508999 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139021c8-a94c-4339-bb0d-442b262cf402 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:05.596275 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced8fecb-e6f9-4afe-9325-861d59fc50d5 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:05.627642 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139021c8-a94c-4339-bb0d-442b262cf402 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:05.646622 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c20e566-2b07-4118-811a-2aee6a6cf9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:05.648133 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20e566-2b07-4118-811a-2aee6a6cf9ba None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:05.648600 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20e566-2b07-4118-811a-2aee6a6cf9ba None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:05.649613 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20e566-2b07-4118-811a-2aee6a6cf9ba None admin] PATH_INFO: `/v3/roles/member` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:05.654239 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:05.666483 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c20e566-2b07-4118-811a-2aee6a6cf9ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:27:05.667709 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279c4545-27fb-430c-aaba-25b77bf05a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:05.668587 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c4545-27fb-430c-aaba-25b77bf05a2a None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:05.668934 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c4545-27fb-430c-aaba-25b77bf05a2a None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:05.669272 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c4545-27fb-430c-aaba-25b77bf05a2a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:05.670609 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279c4545-27fb-430c-aaba-25b77bf05a2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:05.672169 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279c4545-27fb-430c-aaba-25b77bf05a2a None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:05.673905 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 109/213] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:27:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:27:05.676085 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-5c20e566-2b07-4118-811a-2aee6a6cf9ba None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:27:05.676085 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:05.676085 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:05.676085 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:05.676085 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:05.676085 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:05.676085 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:05.676085 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:05.676085 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:05.676085 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:05.676085 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:05.676085 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:05.676085 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:27:05.676085 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:27:05.676085 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:27:05.676085 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:27:05.676948 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:27:05.676948 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:27:05.676948 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:27:05.676948 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:27:05.676948 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:05.676948 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:05.676948 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:05.676948 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:05.676948 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:05.676948 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:05.676948 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:05.676948 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:05.676948 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:05.676948 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:05.676948 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:05.676948 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:05.677735 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:05.677735 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:05.677735 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:05.677735 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:05.677735 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:27:05.677735 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:27:05.677735 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:27:05.677735 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:27:05.677735 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:27:05.677735 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:27:05.677735 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:27:05.677735 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:27:05.679691 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279c4545-27fb-430c-aaba-25b77bf05a2a None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:05.680672 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 105/214] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:27:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:27:05.686838 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c20e566-2b07-4118-811a-2aee6a6cf9ba None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:05.721859 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:05.808123 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c20e566-2b07-4118-811a-2aee6a6cf9ba None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:05.814410 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279c4545-27fb-430c-aaba-25b77bf05a2a None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:05.851681 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be7ef3-3ddb-467c-9b34-4e2175d8a6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:05.852504 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be7ef3-3ddb-467c-9b34-4e2175d8a6ef None admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:05.852856 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be7ef3-3ddb-467c-9b34-4e2175d8a6ef None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:05.853276 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be7ef3-3ddb-467c-9b34-4e2175d8a6ef None admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:05.854111 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37be7ef3-3ddb-467c-9b34-4e2175d8a6ef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:05.855679 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806425a7-0f42-4fdf-9a74-32f24d20fb85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:05.856143 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37be7ef3-3ddb-467c-9b34-4e2175d8a6ef None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:05.856756 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806425a7-0f42-4fdf-9a74-32f24d20fb85 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:05.857230 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806425a7-0f42-4fdf-9a74-32f24d20fb85 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:05.857683 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806425a7-0f42-4fdf-9a74-32f24d20fb85 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:05.858662 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806425a7-0f42-4fdf-9a74-32f24d20fb85 None admin] RBAC: Adding query filter params (name=member) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:05.859153 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806425a7-0f42-4fdf-9a74-32f24d20fb85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:05.860712 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806425a7-0f42-4fdf-9a74-32f24d20fb85 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:05.877892 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 106/215] 10.222.0.13 () {56 vars in 1101 bytes} [Fri Jun 24 08:27:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:27:05.879075 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806425a7-0f42-4fdf-9a74-32f24d20fb85 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:05.880688 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 110/216] 10.222.0.13 () {60 vars in 1132 bytes} [Fri Jun 24 08:27:05 2022] POST /identity/v3/users => generated 297 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:27:05.978772 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:05.989235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a474ed-8696-44dc-9fa7-617e6a3b0a34 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:05.989500 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a474ed-8696-44dc-9fa7-617e6a3b0a34 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:05.989774 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a474ed-8696-44dc-9fa7-617e6a3b0a34 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:05.991446 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 111/217] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:05.997626 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806425a7-0f42-4fdf-9a74-32f24d20fb85 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:05.998136 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fcf2e8-0552-43f8-96b6-a5484c96dffd None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:05.998428 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fcf2e8-0552-43f8-96b6-a5484c96dffd None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:05.998711 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fcf2e8-0552-43f8-96b6-a5484c96dffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:06.017630 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-69fcf2e8-0552-43f8-96b6-a5484c96dffd None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:06.018872 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-69fcf2e8-0552-43f8-96b6-a5484c96dffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:06.037675 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5e467a-9a74-4411-b36c-00945a06cbdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:06.039019 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e467a-9a74-4411-b36c-00945a06cbdd None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:06.039622 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e467a-9a74-4411-b36c-00945a06cbdd None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:06.040081 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e467a-9a74-4411-b36c-00945a06cbdd None admin] PATH_INFO: `/v3/users/eba4bb48555c48eca3b0e1f2995bac9a` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:06.056455 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5e467a-9a74-4411-b36c-00945a06cbdd None admin] RBAC: Authorizing `identity:get_user(user_id=eba4bb48555c48eca3b0e1f2995bac9a)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:06.058074 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5e467a-9a74-4411-b36c-00945a06cbdd None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:06.060130 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 107/218] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Jun 24 08:27:05 2022] GET /identity/v3/users/eba4bb48555c48eca3b0e1f2995bac9a => generated 297 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:06.065506 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5e467a-9a74-4411-b36c-00945a06cbdd None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:06.073666 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:06.105972 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 112/219] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 08:27:06.129774 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fcf2e8-0552-43f8-96b6-a5484c96dffd None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:06.194032 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5e467a-9a74-4411-b36c-00945a06cbdd None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:06.237316 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee1c10e-8605-41d7-ad72-ac5c24c932bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:06.238553 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1c10e-8605-41d7-ad72-ac5c24c932bc None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:06.238852 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1c10e-8605-41d7-ad72-ac5c24c932bc None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:06.239136 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1c10e-8605-41d7-ad72-ac5c24c932bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:06.256324 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:06.272084 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fcf2e8-0552-43f8-96b6-a5484c96dffd None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:06.311213 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75070ddf-3a8b-4fb0-ac46-4962b8d2aa4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:06.312358 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75070ddf-3a8b-4fb0-ac46-4962b8d2aa4e None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:06.312746 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75070ddf-3a8b-4fb0-ac46-4962b8d2aa4e None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:06.313034 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75070ddf-3a8b-4fb0-ac46-4962b8d2aa4e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:06.327581 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75070ddf-3a8b-4fb0-ac46-4962b8d2aa4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:27:06.330783 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee1c10e-8605-41d7-ad72-ac5c24c932bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:06.332282 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee1c10e-8605-41d7-ad72-ac5c24c932bc None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:06.337850 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-75070ddf-3a8b-4fb0-ac46-4962b8d2aa4e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:27:06.337850 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:06.337850 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:06.337850 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:06.337850 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:06.337850 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:06.337850 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:06.337850 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:06.337850 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:06.337850 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:06.337850 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:06.337850 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:06.337850 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:27:06.337850 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:27:06.337850 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:27:06.337850 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:27:06.338781 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:27:06.338781 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:27:06.338781 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:27:06.338781 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:27:06.338781 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:06.338781 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:06.338781 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:06.338781 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:06.338781 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:06.338781 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:06.338781 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:06.338781 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:06.338781 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:06.338781 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:06.338781 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:06.338781 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:06.339627 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:06.339627 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:06.339627 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:06.339627 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:06.339627 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:27:06.339627 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:27:06.339627 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:27:06.339627 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:27:06.339627 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:27:06.339627 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:27:06.339627 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:27:06.339627 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:06.339627 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 113/220] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Jun 24 08:27:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:27:06.340680 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a7ef09-8574-4791-ab83-4b32ac0d0310 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:06.341221 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a7ef09-8574-4791-ab83-4b32ac0d0310 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:06.341221 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a7ef09-8574-4791-ab83-4b32ac0d0310 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:06.346962 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 114/221] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:06.349555 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a7ef09-8574-4791-ab83-4b32ac0d0310 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:06.394828 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:06.469129 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 108/222] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:27:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 08:27:06.470611 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f5dc2-3977-44ff-9739-4105e13a096b None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:06.470808 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f5dc2-3977-44ff-9739-4105e13a096b None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:06.471087 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f5dc2-3977-44ff-9739-4105e13a096b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:06.480401 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a7ef09-8574-4791-ab83-4b32ac0d0310 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:06.489656 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-fc7f5dc2-3977-44ff-9739-4105e13a096b None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:06.490551 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-fc7f5dc2-3977-44ff-9739-4105e13a096b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:06.525356 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1e835b-7907-41c7-9139-850ef43408de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:06.526543 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e835b-7907-41c7-9139-850ef43408de None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:06.527321 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e835b-7907-41c7-9139-850ef43408de None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:06.528084 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e835b-7907-41c7-9139-850ef43408de None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:06.529269 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1e835b-7907-41c7-9139-850ef43408de None admin] RBAC: Adding query filter params (name=admin) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:06.529888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1e835b-7907-41c7-9139-850ef43408de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:06.531486 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1e835b-7907-41c7-9139-850ef43408de None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:06.532737 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:06.542609 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 115/223] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:27:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:06.543940 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1e835b-7907-41c7-9139-850ef43408de None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:06.581949 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 109/224] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:27:06.582846 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7f5dc2-3977-44ff-9739-4105e13a096b None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:06.672597 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:06.679528 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1e835b-7907-41c7-9139-850ef43408de None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:06.707779 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7f5dc2-3977-44ff-9739-4105e13a096b None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:06.726328 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006fada-22ab-4f49-ae9a-b734295dbde6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:06.727810 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006fada-22ab-4f49-ae9a-b734295dbde6 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:06.728269 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006fada-22ab-4f49-ae9a-b734295dbde6 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:06.728715 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006fada-22ab-4f49-ae9a-b734295dbde6 None admin] PATH_INFO: `/v3/projects/58dfc569119141c688e8c2f5c4faab17` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:06.730672 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4006fada-22ab-4f49-ae9a-b734295dbde6 None admin] RBAC: Authorizing `identity:get_project(project_id=58dfc569119141c688e8c2f5c4faab17)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:06.732847 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4006fada-22ab-4f49-ae9a-b734295dbde6 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:06.735954 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 116/225] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:27:06 2022] GET /identity/v3/projects/58dfc569119141c688e8c2f5c4faab17 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:06.737475 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006fada-22ab-4f49-ae9a-b734295dbde6 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:06.745921 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73248f5b-75ef-426a-9494-80868fce7e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:06.746901 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73248f5b-75ef-426a-9494-80868fce7e48 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:06.747314 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73248f5b-75ef-426a-9494-80868fce7e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:06.747664 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73248f5b-75ef-426a-9494-80868fce7e48 None admin] PATH_INFO: `/v3/users/394ee1b1c13f4d2989c71839d7642eb2` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:06.749320 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73248f5b-75ef-426a-9494-80868fce7e48 None admin] RBAC: Authorizing `identity:get_user(user_id=394ee1b1c13f4d2989c71839d7642eb2)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:06.751097 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73248f5b-75ef-426a-9494-80868fce7e48 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:06.752742 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 110/226] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Jun 24 08:27:06 2022] GET /identity/v3/users/394ee1b1c13f4d2989c71839d7642eb2 => generated 283 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:06.753737 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73248f5b-75ef-426a-9494-80868fce7e48 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:06.810372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:06.872717 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73248f5b-75ef-426a-9494-80868fce7e48 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:06.874979 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006fada-22ab-4f49-ae9a-b734295dbde6 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:06.909861 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4b408e-d37e-494b-9c67-f158b9626dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:06.910526 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4b408e-d37e-494b-9c67-f158b9626dd3 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:06.910812 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4b408e-d37e-494b-9c67-f158b9626dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:06.911090 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4b408e-d37e-494b-9c67-f158b9626dd3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:06.911925 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4b408e-d37e-494b-9c67-f158b9626dd3 None admin] RBAC: Adding query filter params (role.id=bd56838b71ec4a5781fe2df9f3f4f1ab, scope.project.id=58dfc569119141c688e8c2f5c4faab17, user.id=eba4bb48555c48eca3b0e1f2995bac9a) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:06.912238 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4b408e-d37e-494b-9c67-f158b9626dd3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:06.913817 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4b408e-d37e-494b-9c67-f158b9626dd3 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:06.918251 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbbb64d-c7a5-4d75-be8a-7c4cfdb72f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:06.919361 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbbb64d-c7a5-4d75-be8a-7c4cfdb72f81 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:06.919846 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbbb64d-c7a5-4d75-be8a-7c4cfdb72f81 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:06.920046 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 111/227] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Jun 24 08:27:06 2022] GET /identity/v3/role_assignments?user.id=eba4bb48555c48eca3b0e1f2995bac9a&scope.project.id=58dfc569119141c688e8c2f5c4faab17&role.id=bd56838b71ec4a5781fe2df9f3f4f1ab => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:06.920541 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbbb64d-c7a5-4d75-be8a-7c4cfdb72f81 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:06.920901 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4b408e-d37e-494b-9c67-f158b9626dd3 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:06.943415 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbbb64d-c7a5-4d75-be8a-7c4cfdb72f81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:27:06.957552 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-dcbbb64d-c7a5-4d75-be8a-7c4cfdb72f81 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 08:27:06.957552 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:06.957552 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:06.957552 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:06.957552 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:06.957552 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:06.957552 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:06.957552 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:06.957552 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:06.957552 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:06.957552 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:06.957552 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:06.957552 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:27:06.957552 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:27:06.957552 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:27:06.957552 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:27:06.958525 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:27:06.958525 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:27:06.958525 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:27:06.958525 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:27:06.958525 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:06.958525 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:06.958525 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:06.958525 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:06.958525 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:06.958525 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:06.958525 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:06.958525 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:06.958525 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:06.958525 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:06.958525 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:06.958525 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:06.959340 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:06.959340 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:06.959340 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:06.959340 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:06.959340 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:27:06.959340 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:27:06.959340 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:27:06.959340 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:27:06.959340 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:27:06.959340 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:27:06.959340 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 08:27:06.959340 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:06.967404 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 117/228] 10.222.0.13 () {56 vars in 1114 bytes} [Fri Jun 24 08:27:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:06.973577 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbbb64d-c7a5-4d75-be8a-7c4cfdb72f81 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:07.027441 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:07.042813 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4b408e-d37e-494b-9c67-f158b9626dd3 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:07.045445 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:07.091200 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ec8e6f-8b22-402d-9776-57e4b2207cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:07.092297 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ec8e6f-8b22-402d-9776-57e4b2207cc1 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:07.092591 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ec8e6f-8b22-402d-9776-57e4b2207cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:07.092871 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ec8e6f-8b22-402d-9776-57e4b2207cc1 None admin] PATH_INFO: `/v3/projects/58dfc569119141c688e8c2f5c4faab17` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:07.096816 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ec8e6f-8b22-402d-9776-57e4b2207cc1 None admin] RBAC: Authorizing `identity:get_project(project_id=58dfc569119141c688e8c2f5c4faab17)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:07.098428 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ec8e6f-8b22-402d-9776-57e4b2207cc1 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:07.100758 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 112/229] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:27:06 2022] GET /identity/v3/projects/58dfc569119141c688e8c2f5c4faab17 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:07.107090 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ec8e6f-8b22-402d-9776-57e4b2207cc1 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:07.186172 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbbb64d-c7a5-4d75-be8a-7c4cfdb72f81 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:07.232416 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b47fc9-fc1f-4718-a0aa-9404f813baf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:07.234301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b47fc9-fc1f-4718-a0aa-9404f813baf6 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:07.237777 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b47fc9-fc1f-4718-a0aa-9404f813baf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:07.238282 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b47fc9-fc1f-4718-a0aa-9404f813baf6 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:07.239242 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b47fc9-fc1f-4718-a0aa-9404f813baf6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:07.239743 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b47fc9-fc1f-4718-a0aa-9404f813baf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:07.241343 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b47fc9-fc1f-4718-a0aa-9404f813baf6 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:07.264109 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 118/230] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:27:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:27:07.274733 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b47fc9-fc1f-4718-a0aa-9404f813baf6 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:07.310230 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ec8e6f-8b22-402d-9776-57e4b2207cc1 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:07.363335 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e81ba9-18a7-4683-ad95-8cd3189bcb38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:07.364241 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e81ba9-18a7-4683-ad95-8cd3189bcb38 None admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:07.364508 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e81ba9-18a7-4683-ad95-8cd3189bcb38 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:07.364776 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e81ba9-18a7-4683-ad95-8cd3189bcb38 None admin] PATH_INFO: `/v3/projects/58dfc569119141c688e8c2f5c4faab17/users/394ee1b1c13f4d2989c71839d7642eb2/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:07.374822 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e81ba9-18a7-4683-ad95-8cd3189bcb38 None admin] RBAC: Authorizing `identity:create_grant(project_id=58dfc569119141c688e8c2f5c4faab17, user_id=394ee1b1c13f4d2989c71839d7642eb2, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:07.376310 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e81ba9-18a7-4683-ad95-8cd3189bcb38 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:07.408233 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 113/231] 10.222.0.13 () {58 vars in 1439 bytes} [Fri Jun 24 08:27:07 2022] PUT /identity/v3/projects/58dfc569119141c688e8c2f5c4faab17/users/394ee1b1c13f4d2989c71839d7642eb2/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:27:07.416158 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:07.428912 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b47fc9-fc1f-4718-a0aa-9404f813baf6 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:07.466865 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c1cf9a-ec6c-487b-806e-fb9d3521aadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:07.467756 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c1cf9a-ec6c-487b-806e-fb9d3521aadb None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:07.468042 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c1cf9a-ec6c-487b-806e-fb9d3521aadb None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:07.468323 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c1cf9a-ec6c-487b-806e-fb9d3521aadb None admin] PATH_INFO: `/v3/users/394ee1b1c13f4d2989c71839d7642eb2` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:07.469569 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c1cf9a-ec6c-487b-806e-fb9d3521aadb None admin] RBAC: Authorizing `identity:get_user(user_id=394ee1b1c13f4d2989c71839d7642eb2)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:07.471060 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c1cf9a-ec6c-487b-806e-fb9d3521aadb None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:07.472905 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 119/232] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Jun 24 08:27:07 2022] GET /identity/v3/users/394ee1b1c13f4d2989c71839d7642eb2 => generated 283 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:27:07.478389 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e81ba9-18a7-4683-ad95-8cd3189bcb38 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:07.521864 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:07.600125 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e81ba9-18a7-4683-ad95-8cd3189bcb38 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:07.636711 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eefdf2a-4c99-4242-b0fa-ec19b9a037c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:07.637573 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eefdf2a-4c99-4242-b0fa-ec19b9a037c4 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:07.637887 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eefdf2a-4c99-4242-b0fa-ec19b9a037c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:07.638160 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eefdf2a-4c99-4242-b0fa-ec19b9a037c4 None admin] PATH_INFO: `/v3/projects/58dfc569119141c688e8c2f5c4faab17` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:07.639362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eefdf2a-4c99-4242-b0fa-ec19b9a037c4 None admin] RBAC: Authorizing `identity:get_project(project_id=58dfc569119141c688e8c2f5c4faab17)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:07.640824 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eefdf2a-4c99-4242-b0fa-ec19b9a037c4 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:07.642909 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 114/233] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:27:07 2022] GET /identity/v3/projects/58dfc569119141c688e8c2f5c4faab17 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:07.648236 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c1cf9a-ec6c-487b-806e-fb9d3521aadb None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:07.755965 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c1cf9a-ec6c-487b-806e-fb9d3521aadb None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:07.794144 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c8210a-88f4-4acd-8863-988fca4e8336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:07.794961 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c8210a-88f4-4acd-8863-988fca4e8336 None admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:07.795249 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c8210a-88f4-4acd-8863-988fca4e8336 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:07.795522 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c8210a-88f4-4acd-8863-988fca4e8336 None admin] PATH_INFO: `/v3/projects/58dfc569119141c688e8c2f5c4faab17/users/394ee1b1c13f4d2989c71839d7642eb2/roles/d6ada62a74a94222b864a6c283f8d9b1` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:07.797857 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c8210a-88f4-4acd-8863-988fca4e8336 None admin] RBAC: Authorizing `identity:create_grant(project_id=58dfc569119141c688e8c2f5c4faab17, user_id=394ee1b1c13f4d2989c71839d7642eb2, role_id=d6ada62a74a94222b864a6c283f8d9b1)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:07.799318 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c8210a-88f4-4acd-8863-988fca4e8336 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:07.832368 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 120/234] 10.222.0.13 () {58 vars in 1439 bytes} [Fri Jun 24 08:27:07 2022] PUT /identity/v3/projects/58dfc569119141c688e8c2f5c4faab17/users/394ee1b1c13f4d2989c71839d7642eb2/roles/d6ada62a74a94222b864a6c283f8d9b1 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:27:08.354106 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df87dcd4-ddfd-4c99-a799-dbf9ca3bb1df None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:08.354497 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df87dcd4-ddfd-4c99-a799-dbf9ca3bb1df None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:08.354669 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df87dcd4-ddfd-4c99-a799-dbf9ca3bb1df None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:08.356065 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 115/235] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:08.361642 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915f246-1bbf-469f-bf00-5b9790988e04 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:08.361959 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915f246-1bbf-469f-bf00-5b9790988e04 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:08.362238 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915f246-1bbf-469f-bf00-5b9790988e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:08.385620 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-5915f246-1bbf-469f-bf00-5b9790988e04 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:08.386852 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-5915f246-1bbf-469f-bf00-5b9790988e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:08.405265 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:08.487393 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 121/236] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:27:08.532208 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df87dcd4-ddfd-4c99-a799-dbf9ca3bb1df None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:08.545915 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c9db04-9f8a-4c43-aec4-addbb292ba19 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:08.546424 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c9db04-9f8a-4c43-aec4-addbb292ba19 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:08.546880 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c9db04-9f8a-4c43-aec4-addbb292ba19 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:08.548691 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 122/237] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:27:08.553645 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4da67-87aa-4a8f-a1d0-83094da0b4ee None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:08.554133 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4da67-87aa-4a8f-a1d0-83094da0b4ee None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:08.554557 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4da67-87aa-4a8f-a1d0-83094da0b4ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:08.590834 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-78e4da67-87aa-4a8f-a1d0-83094da0b4ee None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:08.592289 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-78e4da67-87aa-4a8f-a1d0-83094da0b4ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:08.649853 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df87dcd4-ddfd-4c99-a799-dbf9ca3bb1df None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:08.654071 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:08.687167 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2709c58d-5618-4246-9e41-45106a2fb987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:08.688248 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2709c58d-5618-4246-9e41-45106a2fb987 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:08.688976 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2709c58d-5618-4246-9e41-45106a2fb987 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:08.689421 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2709c58d-5618-4246-9e41-45106a2fb987 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:08.704094 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2709c58d-5618-4246-9e41-45106a2fb987 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:27:08.712623 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-2709c58d-5618-4246-9e41-45106a2fb987 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 08:27:08.712623 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:08.712623 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:08.712623 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:08.712623 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:08.712623 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:08.712623 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:08.712623 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:08.712623 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:08.712623 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:08.712623 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:08.712623 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:08.712623 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:27:08.712623 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:27:08.712623 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:27:08.712623 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:27:08.713516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:27:08.713516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:27:08.713516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:27:08.713516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:27:08.713516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:08.713516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:08.713516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:08.713516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:08.713516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:08.713516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:08.713516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:08.713516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:08.713516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:08.713516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:08.713516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:08.713516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:08.714327 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:08.714327 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:08.714327 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:08.714327 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:08.714327 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:27:08.714327 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:27:08.714327 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:27:08.714327 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:27:08.714327 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:27:08.714327 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:27:08.714327 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 08:27:08.714327 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:27:08.716397 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 116/238] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:27:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:08.726675 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2709c58d-5618-4246-9e41-45106a2fb987 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:08.747090 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 123/239] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 08:27:08.786779 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e4da67-87aa-4a8f-a1d0-83094da0b4ee None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:08.833523 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2709c58d-5618-4246-9e41-45106a2fb987 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:08.869975 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edac2ca-2a8e-423f-8b2d-48aac6ed3eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:08.871048 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edac2ca-2a8e-423f-8b2d-48aac6ed3eb0 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:08.871524 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edac2ca-2a8e-423f-8b2d-48aac6ed3eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:08.871958 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edac2ca-2a8e-423f-8b2d-48aac6ed3eb0 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:08.872909 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edac2ca-2a8e-423f-8b2d-48aac6ed3eb0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:08.873404 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edac2ca-2a8e-423f-8b2d-48aac6ed3eb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:08.875269 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edac2ca-2a8e-423f-8b2d-48aac6ed3eb0 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:08.885292 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 117/240] 10.222.0.13 () {56 vars in 1101 bytes} [Fri Jun 24 08:27:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:08.891114 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edac2ca-2a8e-423f-8b2d-48aac6ed3eb0 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:08.946505 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:08.946854 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:08.972221 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e4da67-87aa-4a8f-a1d0-83094da0b4ee None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:09.000322 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edac2ca-2a8e-423f-8b2d-48aac6ed3eb0 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:09.019772 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5f9115-bf2f-43b8-bae8-bd8b0b0e7ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:09.020591 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f9115-bf2f-43b8-bae8-bd8b0b0e7ff4 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:09.020875 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f9115-bf2f-43b8-bae8-bd8b0b0e7ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:09.021152 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f9115-bf2f-43b8-bae8-bd8b0b0e7ff4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:09.028133 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5f9115-bf2f-43b8-bae8-bd8b0b0e7ff4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:09.028133 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5f9115-bf2f-43b8-bae8-bd8b0b0e7ff4 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:09.030319 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 124/241] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:27:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:09.038773 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5f9115-bf2f-43b8-bae8-bd8b0b0e7ff4 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:09.047037 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b11a481-fd55-4639-95ad-abb76c484831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:09.048256 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b11a481-fd55-4639-95ad-abb76c484831 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:09.048779 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b11a481-fd55-4639-95ad-abb76c484831 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:09.049233 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b11a481-fd55-4639-95ad-abb76c484831 None admin] PATH_INFO: `/v3/users/efc8f14f69c74fb4a15e7b37360af9c5` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:09.083091 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b11a481-fd55-4639-95ad-abb76c484831 None admin] RBAC: Authorizing `identity:get_user(user_id=efc8f14f69c74fb4a15e7b37360af9c5)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:09.084855 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b11a481-fd55-4639-95ad-abb76c484831 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:09.087852 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 118/242] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Jun 24 08:27:08 2022] GET /identity/v3/users/efc8f14f69c74fb4a15e7b37360af9c5 => generated 297 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:27:09.095997 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b11a481-fd55-4639-95ad-abb76c484831 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:09.230161 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:09.231340 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5f9115-bf2f-43b8-bae8-bd8b0b0e7ff4 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:09.266138 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b11a481-fd55-4639-95ad-abb76c484831 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:09.292773 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30010af0-c430-4c9b-b55d-3c3673b7f7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:09.293449 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30010af0-c430-4c9b-b55d-3c3673b7f7dd None admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:09.293747 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30010af0-c430-4c9b-b55d-3c3673b7f7dd None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:09.294022 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30010af0-c430-4c9b-b55d-3c3673b7f7dd None admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:09.294733 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30010af0-c430-4c9b-b55d-3c3673b7f7dd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:09.296152 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30010af0-c430-4c9b-b55d-3c3673b7f7dd None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:09.323322 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3252e3aa-aafe-4172-ba2d-95756234dcb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:09.323972 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3252e3aa-aafe-4172-ba2d-95756234dcb8 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:09.324259 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3252e3aa-aafe-4172-ba2d-95756234dcb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:09.324576 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3252e3aa-aafe-4172-ba2d-95756234dcb8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:09.330490 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:09.335195 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 125/243] 10.222.0.13 () {60 vars in 1132 bytes} [Fri Jun 24 08:27:09 2022] POST /identity/v3/users => generated 293 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:27:09.440203 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3252e3aa-aafe-4172-ba2d-95756234dcb8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:09.441723 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3252e3aa-aafe-4172-ba2d-95756234dcb8 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:09.545562 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:09.572008 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 119/244] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:27:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:27:09.577733 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30010af0-c430-4c9b-b55d-3c3673b7f7dd None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:09.627884 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd4e116-57e2-44d5-b864-1653a40106b6 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:09.628229 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd4e116-57e2-44d5-b864-1653a40106b6 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:09.628452 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd4e116-57e2-44d5-b864-1653a40106b6 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:09.629886 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 120/245] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:09.638354 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63569c-6060-4cc2-8bf2-01716ca2a356 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:09.638643 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63569c-6060-4cc2-8bf2-01716ca2a356 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:09.638931 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63569c-6060-4cc2-8bf2-01716ca2a356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:09.660149 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-ba63569c-6060-4cc2-8bf2-01716ca2a356 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:09.661340 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-ba63569c-6060-4cc2-8bf2-01716ca2a356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:09.709625 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30010af0-c430-4c9b-b55d-3c3673b7f7dd None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:09.757667 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1783f1-5322-411c-8f06-01d37cb82afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:09.759142 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 121/246] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 08:27:09.759681 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1783f1-5322-411c-8f06-01d37cb82afa None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:09.759967 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1783f1-5322-411c-8f06-01d37cb82afa None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:09.760308 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1783f1-5322-411c-8f06-01d37cb82afa None admin] PATH_INFO: `/v3/projects/58dfc569119141c688e8c2f5c4faab17` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:09.761579 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1783f1-5322-411c-8f06-01d37cb82afa None admin] RBAC: Authorizing `identity:get_project(project_id=58dfc569119141c688e8c2f5c4faab17)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:09.763074 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1783f1-5322-411c-8f06-01d37cb82afa None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:09.764944 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 126/247] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:27:09 2022] GET /identity/v3/projects/58dfc569119141c688e8c2f5c4faab17 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:09.770498 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba63569c-6060-4cc2-8bf2-01716ca2a356 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:09.785420 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1783f1-5322-411c-8f06-01d37cb82afa None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:09.881205 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:09.896334 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba63569c-6060-4cc2-8bf2-01716ca2a356 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:09.910786 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:09.934979 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1783f1-5322-411c-8f06-01d37cb82afa None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:09.941467 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3681a27-3360-48f7-a7f7-e52de4fe4063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:09.942185 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3681a27-3360-48f7-a7f7-e52de4fe4063 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:09.942473 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3681a27-3360-48f7-a7f7-e52de4fe4063 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:09.942754 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3681a27-3360-48f7-a7f7-e52de4fe4063 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:09.943597 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3681a27-3360-48f7-a7f7-e52de4fe4063 None admin] RBAC: Adding query filter params (role.id=986a50f3abf14fdab55012a8e52719c7, scope.project.id=58dfc569119141c688e8c2f5c4faab17, user.id=efc8f14f69c74fb4a15e7b37360af9c5) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:09.943989 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3681a27-3360-48f7-a7f7-e52de4fe4063 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:09.946190 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3681a27-3360-48f7-a7f7-e52de4fe4063 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:09.954106 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 122/248] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Jun 24 08:27:09 2022] GET /identity/v3/role_assignments?user.id=efc8f14f69c74fb4a15e7b37360af9c5&scope.project.id=58dfc569119141c688e8c2f5c4faab17&role.id=986a50f3abf14fdab55012a8e52719c7 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:09.977138 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c2e1d1-6343-4dc7-8360-c686c6e07e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:09.978472 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c2e1d1-6343-4dc7-8360-c686c6e07e98 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:09.978924 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c2e1d1-6343-4dc7-8360-c686c6e07e98 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:09.979341 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c2e1d1-6343-4dc7-8360-c686c6e07e98 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:09.992333 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c2e1d1-6343-4dc7-8360-c686c6e07e98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:27:10.001427 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-12c2e1d1-6343-4dc7-8360-c686c6e07e98 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:27:10.001427 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:10.001427 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:10.001427 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:10.001427 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:10.001427 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:10.001427 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:10.001427 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:10.001427 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:10.001427 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:10.001427 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:10.001427 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:10.001427 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:27:10.001427 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:27:10.001427 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:27:10.001427 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:27:10.002573 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:27:10.002573 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:27:10.002573 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:27:10.002573 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:27:10.002573 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:10.002573 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:10.002573 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:10.002573 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:10.002573 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:10.002573 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:10.002573 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:10.002573 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:10.002573 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:10.002573 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:10.002573 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:10.002573 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:10.003493 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:10.003493 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:10.003493 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:10.003493 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:10.003493 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:27:10.003493 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:27:10.003493 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:27:10.003493 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:27:10.003493 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:27:10.003493 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:27:10.003493 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:27:10.003493 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:10.005568 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 127/249] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:27:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:27:10.012286 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3681a27-3360-48f7-a7f7-e52de4fe4063 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:10.140877 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3681a27-3360-48f7-a7f7-e52de4fe4063 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:10.182736 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2f50b3-5344-4b63-8baa-200f45951b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:10.182964 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f50b3-5344-4b63-8baa-200f45951b3c None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:10.183230 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f50b3-5344-4b63-8baa-200f45951b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:10.183510 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f50b3-5344-4b63-8baa-200f45951b3c None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:10.185133 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2f50b3-5344-4b63-8baa-200f45951b3c None admin] RBAC: Adding query filter params (name=member) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:10.185133 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2f50b3-5344-4b63-8baa-200f45951b3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:10.185987 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2f50b3-5344-4b63-8baa-200f45951b3c None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:10.197186 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 123/250] 10.222.0.13 () {56 vars in 1101 bytes} [Fri Jun 24 08:27:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:10.202750 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c2e1d1-6343-4dc7-8360-c686c6e07e98 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:10.329037 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c2e1d1-6343-4dc7-8360-c686c6e07e98 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:10.376392 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090070f5-9320-4c08-b99a-b5b30d0d9f0e None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:10.376972 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090070f5-9320-4c08-b99a-b5b30d0d9f0e None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:10.377427 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090070f5-9320-4c08-b99a-b5b30d0d9f0e None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:10.379057 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 124/251] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:10.383608 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:10.391253 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99483c73-db21-4475-bcbb-e3da30311647 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:10.391802 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99483c73-db21-4475-bcbb-e3da30311647 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:10.392277 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99483c73-db21-4475-bcbb-e3da30311647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:10.413332 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b473d2e7-9b70-4aff-99c4-e2de66de86d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:10.415111 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b473d2e7-9b70-4aff-99c4-e2de66de86d0 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:10.415581 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b473d2e7-9b70-4aff-99c4-e2de66de86d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:10.416034 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b473d2e7-9b70-4aff-99c4-e2de66de86d0 None admin] PATH_INFO: `/v3/users/eba4bb48555c48eca3b0e1f2995bac9a` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:10.417772 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-99483c73-db21-4475-bcbb-e3da30311647 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:10.418613 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b473d2e7-9b70-4aff-99c4-e2de66de86d0 None admin] RBAC: Authorizing `identity:get_user(user_id=eba4bb48555c48eca3b0e1f2995bac9a)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:10.421060 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b473d2e7-9b70-4aff-99c4-e2de66de86d0 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:10.422564 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-99483c73-db21-4475-bcbb-e3da30311647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:10.426928 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 128/252] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Jun 24 08:27:10 2022] GET /identity/v3/users/eba4bb48555c48eca3b0e1f2995bac9a => generated 297 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:10.435261 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b473d2e7-9b70-4aff-99c4-e2de66de86d0 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:10.485366 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:10.544903 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 125/253] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:27:10.579043 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99483c73-db21-4475-bcbb-e3da30311647 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:10.594830 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b473d2e7-9b70-4aff-99c4-e2de66de86d0 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:10.666537 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68226750-ac46-4246-9924-29e5cc5b7661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:10.668287 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68226750-ac46-4246-9924-29e5cc5b7661 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:10.668579 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68226750-ac46-4246-9924-29e5cc5b7661 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:10.668856 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68226750-ac46-4246-9924-29e5cc5b7661 None admin] PATH_INFO: `/v3/projects/58dfc569119141c688e8c2f5c4faab17` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:10.670607 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68226750-ac46-4246-9924-29e5cc5b7661 None admin] RBAC: Authorizing `identity:get_project(project_id=58dfc569119141c688e8c2f5c4faab17)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:10.672964 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68226750-ac46-4246-9924-29e5cc5b7661 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:10.677724 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 129/254] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:27:10 2022] GET /identity/v3/projects/58dfc569119141c688e8c2f5c4faab17 => generated 302 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:10.682965 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68226750-ac46-4246-9924-29e5cc5b7661 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:10.778597 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:10.798408 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99483c73-db21-4475-bcbb-e3da30311647 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:10.860400 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68226750-ac46-4246-9924-29e5cc5b7661 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:10.871819 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dba800-b1da-4763-a417-f40cbab07a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:10.872636 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dba800-b1da-4763-a417-f40cbab07a24 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:10.872919 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dba800-b1da-4763-a417-f40cbab07a24 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:10.873196 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dba800-b1da-4763-a417-f40cbab07a24 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:10.902106 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9051a977-cd85-434d-89f0-b0e5f6671b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:10.903377 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9051a977-cd85-434d-89f0-b0e5f6671b15 None admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:10.903843 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9051a977-cd85-434d-89f0-b0e5f6671b15 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:10.904296 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9051a977-cd85-434d-89f0-b0e5f6671b15 None admin] PATH_INFO: `/v3/projects/58dfc569119141c688e8c2f5c4faab17/users/eba4bb48555c48eca3b0e1f2995bac9a/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:10.906609 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dba800-b1da-4763-a417-f40cbab07a24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:27:10.906909 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9051a977-cd85-434d-89f0-b0e5f6671b15 None admin] RBAC: Authorizing `identity:create_grant(project_id=58dfc569119141c688e8c2f5c4faab17, user_id=eba4bb48555c48eca3b0e1f2995bac9a, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:10.908561 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9051a977-cd85-434d-89f0-b0e5f6671b15 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:10.914396 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-e5dba800-b1da-4763-a417-f40cbab07a24 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:27:10.914396 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:10.914396 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:10.914396 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:10.914396 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:10.914396 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:10.914396 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:10.914396 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:10.914396 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:10.914396 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:10.914396 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:10.914396 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:10.914396 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:27:10.914396 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:27:10.914396 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:27:10.914396 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:27:10.915310 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:27:10.915310 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:27:10.915310 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:27:10.915310 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:27:10.915310 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:10.915310 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:10.915310 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:10.915310 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:10.915310 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:10.915310 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:10.915310 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:10.915310 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:10.915310 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:10.915310 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:10.915310 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:10.915310 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:10.916292 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:10.916292 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:10.916292 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:10.916292 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:10.916292 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:27:10.916292 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:27:10.916292 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:27:10.916292 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:27:10.916292 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:27:10.916292 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:27:10.916292 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:27:10.916292 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:27:10.917827 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 126/255] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Jun 24 08:27:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:27:10.926701 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 130/256] 10.222.0.13 () {58 vars in 1439 bytes} [Fri Jun 24 08:27:10 2022] PUT /identity/v3/projects/58dfc569119141c688e8c2f5c4faab17/users/eba4bb48555c48eca3b0e1f2995bac9a/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:27:10.927882 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dba800-b1da-4763-a417-f40cbab07a24 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:10.948855 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2650bdd3-549c-4790-a3e3-980e9845366a None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:10.949433 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2650bdd3-549c-4790-a3e3-980e9845366a None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:10.949881 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2650bdd3-549c-4790-a3e3-980e9845366a None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:10.951811 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 131/257] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:10.958254 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8e2db-7a64-4ff0-ac1b-f542effe923f None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:10.958735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8e2db-7a64-4ff0-ac1b-f542effe923f None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:10.959177 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8e2db-7a64-4ff0-ac1b-f542effe923f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:10.993463 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:10.995546 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-91d8e2db-7a64-4ff0-ac1b-f542effe923f None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:10.996838 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-91d8e2db-7a64-4ff0-ac1b-f542effe923f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:11.085932 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dba800-b1da-4763-a417-f40cbab07a24 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:11.126682 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527463a8-c7fa-4969-863b-4b2da92caaaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:11.127636 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527463a8-c7fa-4969-863b-4b2da92caaaf None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:11.128311 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527463a8-c7fa-4969-863b-4b2da92caaaf None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:11.128763 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527463a8-c7fa-4969-863b-4b2da92caaaf None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:11.129681 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527463a8-c7fa-4969-863b-4b2da92caaaf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:11.130234 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527463a8-c7fa-4969-863b-4b2da92caaaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:11.131872 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527463a8-c7fa-4969-863b-4b2da92caaaf None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:11.145822 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 127/258] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:27:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:11.155225 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527463a8-c7fa-4969-863b-4b2da92caaaf None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:11.255727 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 132/259] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 08:27:11.266145 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:11.276034 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527463a8-c7fa-4969-863b-4b2da92caaaf None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:11.287023 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d8e2db-7a64-4ff0-ac1b-f542effe923f None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:11.316076 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c6f3b-a57e-4b14-b81b-e2232f44d51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:11.317444 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c6f3b-a57e-4b14-b81b-e2232f44d51b None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:11.317821 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c6f3b-a57e-4b14-b81b-e2232f44d51b None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:11.318109 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c6f3b-a57e-4b14-b81b-e2232f44d51b None admin] PATH_INFO: `/v3/users/394ee1b1c13f4d2989c71839d7642eb2` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:11.319370 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3c6f3b-a57e-4b14-b81b-e2232f44d51b None admin] RBAC: Authorizing `identity:get_user(user_id=394ee1b1c13f4d2989c71839d7642eb2)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:11.321174 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3c6f3b-a57e-4b14-b81b-e2232f44d51b None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:11.323329 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 128/260] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Jun 24 08:27:11 2022] GET /identity/v3/users/394ee1b1c13f4d2989c71839d7642eb2 => generated 283 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:11.330744 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c6f3b-a57e-4b14-b81b-e2232f44d51b None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:11.382256 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:11.459031 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d8e2db-7a64-4ff0-ac1b-f542effe923f None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:11.468213 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c6f3b-a57e-4b14-b81b-e2232f44d51b None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:11.506866 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72b517e-6c49-4fc0-91d3-6c1e2302163e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:11.508217 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72b517e-6c49-4fc0-91d3-6c1e2302163e None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:11.508518 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72b517e-6c49-4fc0-91d3-6c1e2302163e None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:11.508846 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72b517e-6c49-4fc0-91d3-6c1e2302163e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:11.539062 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:11.578955 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2167a247-253e-449a-a3f6-57c0e716a812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:11.579770 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2167a247-253e-449a-a3f6-57c0e716a812 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:11.580052 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2167a247-253e-449a-a3f6-57c0e716a812 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:11.580329 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2167a247-253e-449a-a3f6-57c0e716a812 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:11.583415 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72b517e-6c49-4fc0-91d3-6c1e2302163e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:11.584898 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72b517e-6c49-4fc0-91d3-6c1e2302163e None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:11.606611 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2167a247-253e-449a-a3f6-57c0e716a812 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:27:11.622793 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-2167a247-253e-449a-a3f6-57c0e716a812 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 08:27:11.622793 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:11.622793 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:11.622793 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:11.622793 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:11.622793 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:11.622793 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:11.622793 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:11.622793 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:11.622793 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:11.622793 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:11.622793 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:11.622793 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:27:11.622793 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:27:11.622793 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:27:11.622793 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:27:11.623703 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:27:11.623703 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:27:11.623703 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:27:11.623703 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:27:11.623703 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:11.623703 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:11.623703 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:11.623703 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:11.623703 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:11.623703 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:11.623703 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:11.623703 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:11.623703 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:11.623703 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:11.623703 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:11.623703 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:11.624720 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:11.624720 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:11.624720 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:11.624720 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:11.624720 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:27:11.624720 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:27:11.624720 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:27:11.624720 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:27:11.624720 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:27:11.624720 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:27:11.624720 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 08:27:11.624720 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:11.624720 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 133/261] 10.222.0.13 () {56 vars in 1114 bytes} [Fri Jun 24 08:27:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:27:11.630090 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2167a247-253e-449a-a3f6-57c0e716a812 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:11.688289 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 129/262] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:27:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:27:11.693831 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72b517e-6c49-4fc0-91d3-6c1e2302163e None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:11.756013 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:11.803533 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72b517e-6c49-4fc0-91d3-6c1e2302163e None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:11.826601 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2167a247-253e-449a-a3f6-57c0e716a812 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:11.845715 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5609fde0-8fda-41e8-9947-8d25eea09de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:11.845715 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5609fde0-8fda-41e8-9947-8d25eea09de2 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:11.845715 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5609fde0-8fda-41e8-9947-8d25eea09de2 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:11.845715 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5609fde0-8fda-41e8-9947-8d25eea09de2 None admin] PATH_INFO: `/v3/projects/58dfc569119141c688e8c2f5c4faab17` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:11.846093 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5609fde0-8fda-41e8-9947-8d25eea09de2 None admin] RBAC: Authorizing `identity:get_project(project_id=58dfc569119141c688e8c2f5c4faab17)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:11.846093 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5609fde0-8fda-41e8-9947-8d25eea09de2 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:11.847060 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 130/263] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:27:11 2022] GET /identity/v3/projects/58dfc569119141c688e8c2f5c4faab17 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:27:11.854053 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5609fde0-8fda-41e8-9947-8d25eea09de2 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:11.858791 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:11.899899 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dcc399-aef3-4339-8459-09189b3c8eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:11.900593 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dcc399-aef3-4339-8459-09189b3c8eb8 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:11.900880 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dcc399-aef3-4339-8459-09189b3c8eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:11.901155 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dcc399-aef3-4339-8459-09189b3c8eb8 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:11.901933 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dcc399-aef3-4339-8459-09189b3c8eb8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:11.902241 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dcc399-aef3-4339-8459-09189b3c8eb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:11.903625 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dcc399-aef3-4339-8459-09189b3c8eb8 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:11.919860 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 134/264] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:27:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:27:11.925116 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dcc399-aef3-4339-8459-09189b3c8eb8 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:11.961237 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5609fde0-8fda-41e8-9947-8d25eea09de2 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:11.990055 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:12.002368 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36e6b7a-3c2a-48d4-8840-bb6d6208b59d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:12.003423 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e6b7a-3c2a-48d4-8840-bb6d6208b59d None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:12.003712 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e6b7a-3c2a-48d4-8840-bb6d6208b59d None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:12.004044 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e6b7a-3c2a-48d4-8840-bb6d6208b59d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:12.006350 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36e6b7a-3c2a-48d4-8840-bb6d6208b59d None admin] RBAC: Adding query filter params (role.id=acfe707eee6944ad94d1915fc9ee7ea3, scope.project.id=58dfc569119141c688e8c2f5c4faab17, user.id=394ee1b1c13f4d2989c71839d7642eb2) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:12.006350 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36e6b7a-3c2a-48d4-8840-bb6d6208b59d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:12.006820 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36e6b7a-3c2a-48d4-8840-bb6d6208b59d None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:12.014087 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 131/265] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Jun 24 08:27:11 2022] GET /identity/v3/role_assignments?user.id=394ee1b1c13f4d2989c71839d7642eb2&scope.project.id=58dfc569119141c688e8c2f5c4faab17&role.id=acfe707eee6944ad94d1915fc9ee7ea3 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:12.136215 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dcc399-aef3-4339-8459-09189b3c8eb8 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:12.177298 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df97e28-af4c-4f2d-a695-9ef74b590ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:12.178589 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df97e28-af4c-4f2d-a695-9ef74b590ceb None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:12.179073 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df97e28-af4c-4f2d-a695-9ef74b590ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:12.179540 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df97e28-af4c-4f2d-a695-9ef74b590ceb None admin] PATH_INFO: `/v3/users/394ee1b1c13f4d2989c71839d7642eb2` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:12.181642 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df97e28-af4c-4f2d-a695-9ef74b590ceb None admin] RBAC: Authorizing `identity:get_user(user_id=394ee1b1c13f4d2989c71839d7642eb2)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:12.183402 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df97e28-af4c-4f2d-a695-9ef74b590ceb None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:12.185736 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 135/266] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Jun 24 08:27:11 2022] GET /identity/v3/users/394ee1b1c13f4d2989c71839d7642eb2 => generated 283 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:27:12.191012 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36e6b7a-3c2a-48d4-8840-bb6d6208b59d None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:12.309960 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36e6b7a-3c2a-48d4-8840-bb6d6208b59d None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:12.345573 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b025de2-19de-48a6-ad04-0db22b44a8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:12.346318 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b025de2-19de-48a6-ad04-0db22b44a8f6 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:12.346609 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b025de2-19de-48a6-ad04-0db22b44a8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:12.346891 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b025de2-19de-48a6-ad04-0db22b44a8f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:12.417403 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:12.430189 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b025de2-19de-48a6-ad04-0db22b44a8f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:12.431699 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b025de2-19de-48a6-ad04-0db22b44a8f6 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:12.537961 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 132/267] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:27:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:27:12.543654 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df97e28-af4c-4f2d-a695-9ef74b590ceb None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:12.643992 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:12.740059 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24782354-3951-4733-8932-ac33eac6a0f0 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:12.740559 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24782354-3951-4733-8932-ac33eac6a0f0 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:12.740559 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24782354-3951-4733-8932-ac33eac6a0f0 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:12.742157 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 133/268] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:12.746972 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df97e28-af4c-4f2d-a695-9ef74b590ceb None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:12.747797 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadba498-a64e-4900-a094-b25dd4ee8cdd None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:12.748085 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadba498-a64e-4900-a094-b25dd4ee8cdd None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:12.748365 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadba498-a64e-4900-a094-b25dd4ee8cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:12.781900 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-cadba498-a64e-4900-a094-b25dd4ee8cdd None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:12.783100 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-cadba498-a64e-4900-a094-b25dd4ee8cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:12.819848 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8afccef-ed6c-44f4-9bc7-6d1a8cf11567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:12.820983 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8afccef-ed6c-44f4-9bc7-6d1a8cf11567 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:12.821284 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8afccef-ed6c-44f4-9bc7-6d1a8cf11567 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:12.821567 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8afccef-ed6c-44f4-9bc7-6d1a8cf11567 None admin] PATH_INFO: `/v3/projects/58dfc569119141c688e8c2f5c4faab17` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:12.823563 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8afccef-ed6c-44f4-9bc7-6d1a8cf11567 None admin] RBAC: Authorizing `identity:get_project(project_id=58dfc569119141c688e8c2f5c4faab17)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:12.825024 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8afccef-ed6c-44f4-9bc7-6d1a8cf11567 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:12.826963 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 136/269] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:27:12 2022] GET /identity/v3/projects/58dfc569119141c688e8c2f5c4faab17 => generated 302 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:12.832526 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8afccef-ed6c-44f4-9bc7-6d1a8cf11567 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:12.872909 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 134/270] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 08:27:12.898027 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadba498-a64e-4900-a094-b25dd4ee8cdd None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:12.918072 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:13.010262 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadba498-a64e-4900-a094-b25dd4ee8cdd None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:13.011675 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8afccef-ed6c-44f4-9bc7-6d1a8cf11567 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:13.025618 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:13.049637 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30ce733-0f9b-4994-a04b-330cc15a64d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:13.050086 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e85a91-e6c2-4a50-93b1-f5c0e5cb7a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:13.050459 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ce733-0f9b-4994-a04b-330cc15a64d8 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:13.050735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e85a91-e6c2-4a50-93b1-f5c0e5cb7a80 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:13.050831 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ce733-0f9b-4994-a04b-330cc15a64d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:13.051005 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ce733-0f9b-4994-a04b-330cc15a64d8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:13.051097 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e85a91-e6c2-4a50-93b1-f5c0e5cb7a80 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:13.051280 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e85a91-e6c2-4a50-93b1-f5c0e5cb7a80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:13.052107 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e85a91-e6c2-4a50-93b1-f5c0e5cb7a80 None admin] RBAC: Adding query filter params (role.id=d6ada62a74a94222b864a6c283f8d9b1, scope.project.id=58dfc569119141c688e8c2f5c4faab17, user.id=394ee1b1c13f4d2989c71839d7642eb2) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:13.052417 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e85a91-e6c2-4a50-93b1-f5c0e5cb7a80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:13.053879 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e85a91-e6c2-4a50-93b1-f5c0e5cb7a80 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:13.060437 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 137/271] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Jun 24 08:27:12 2022] GET /identity/v3/role_assignments?user.id=394ee1b1c13f4d2989c71839d7642eb2&scope.project.id=58dfc569119141c688e8c2f5c4faab17&role.id=d6ada62a74a94222b864a6c283f8d9b1 => generated 608 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:13.061479 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d91103-1bdf-485b-b704-0ca19dc24adb None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:13.061795 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d91103-1bdf-485b-b704-0ca19dc24adb None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:13.062076 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d91103-1bdf-485b-b704-0ca19dc24adb None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:13.063419 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 138/272] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:13.064431 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4dae1d-207e-4d59-8eb8-67c7e045aa64 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:13.064719 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4dae1d-207e-4d59-8eb8-67c7e045aa64 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:13.064992 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4dae1d-207e-4d59-8eb8-67c7e045aa64 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:13.066463 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 139/273] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:13.074059 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2a1d2-d136-4933-ada3-7a90a5bb6d3f None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:13.074059 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2a1d2-d136-4933-ada3-7a90a5bb6d3f None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:13.074059 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2a1d2-d136-4933-ada3-7a90a5bb6d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:13.075378 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30ce733-0f9b-4994-a04b-330cc15a64d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:27:13.083389 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-f30ce733-0f9b-4994-a04b-330cc15a64d8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:27:13.083389 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:13.083389 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:13.083389 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:13.083389 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:13.083389 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:13.083389 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:13.083389 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:13.083389 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:13.083389 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:13.083389 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:13.083389 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:13.083389 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:27:13.083389 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:27:13.083389 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:27:13.083389 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:27:13.084322 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:27:13.084322 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:27:13.084322 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:27:13.084322 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:27:13.084322 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:13.084322 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:13.084322 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:13.084322 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:13.084322 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:13.084322 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:13.084322 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:13.084322 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:13.084322 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:13.084322 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:13.084322 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:13.084322 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:13.085103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:13.085103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:13.085103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:13.085103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:13.085103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:27:13.085103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:27:13.085103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:27:13.085103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:27:13.085103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:27:13.085103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:27:13.085103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:27:13.085103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:27:13.090058 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 135/274] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:27:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:27:13.092136 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db7a191-33ed-4767-88de-27e4d1b36475 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:13.092421 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db7a191-33ed-4767-88de-27e4d1b36475 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:13.092693 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db7a191-33ed-4767-88de-27e4d1b36475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:13.105147 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-e5e2a1d2-d136-4933-ada3-7a90a5bb6d3f None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:13.106983 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-e5e2a1d2-d136-4933-ada3-7a90a5bb6d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:13.113443 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-4db7a191-33ed-4767-88de-27e4d1b36475 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:13.114979 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-4db7a191-33ed-4767-88de-27e4d1b36475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:13.189812 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 140/275] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:27:13.191006 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e2a1d2-d136-4933-ada3-7a90a5bb6d3f None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:13.220526 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 136/276] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:27:13.223688 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db7a191-33ed-4767-88de-27e4d1b36475 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:13.246135 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:13.339396 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e2a1d2-d136-4933-ada3-7a90a5bb6d3f None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:13.348239 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:13.369667 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db7a191-33ed-4767-88de-27e4d1b36475 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:13.387814 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be958a34-6d53-40c3-9e9d-349155a70d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:13.388479 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be958a34-6d53-40c3-9e9d-349155a70d8b None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:13.388761 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be958a34-6d53-40c3-9e9d-349155a70d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:13.389033 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be958a34-6d53-40c3-9e9d-349155a70d8b None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:13.389815 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be958a34-6d53-40c3-9e9d-349155a70d8b None admin] RBAC: Adding query filter params (name=member) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:13.390123 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be958a34-6d53-40c3-9e9d-349155a70d8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:13.391485 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be958a34-6d53-40c3-9e9d-349155a70d8b None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:13.400679 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 141/277] 10.222.0.13 () {56 vars in 1101 bytes} [Fri Jun 24 08:27:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:13.401460 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be958a34-6d53-40c3-9e9d-349155a70d8b None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:13.409766 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77021d16-7a30-4b43-b1fe-8e8ac3489e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:13.410554 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77021d16-7a30-4b43-b1fe-8e8ac3489e60 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:13.410834 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77021d16-7a30-4b43-b1fe-8e8ac3489e60 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:13.411110 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77021d16-7a30-4b43-b1fe-8e8ac3489e60 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:13.422944 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77021d16-7a30-4b43-b1fe-8e8ac3489e60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:27:13.431848 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-77021d16-7a30-4b43-b1fe-8e8ac3489e60 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 08:27:13.431848 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:13.431848 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:13.431848 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:13.431848 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:13.431848 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:13.431848 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:13.431848 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:13.431848 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:13.431848 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:13.431848 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:13.431848 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:13.431848 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:27:13.431848 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:27:13.431848 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:27:13.431848 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:27:13.432756 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:27:13.432756 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:27:13.432756 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:27:13.432756 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:27:13.432756 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:13.432756 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:13.432756 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:13.432756 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:13.432756 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:13.432756 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:13.432756 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:13.432756 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:13.432756 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:13.432756 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:13.432756 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:13.432756 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:13.433521 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:13.433521 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:13.433521 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:13.433521 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:13.433521 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:27:13.433521 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:27:13.433521 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:27:13.433521 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:27:13.433521 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:27:13.433521 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:27:13.433521 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 08:27:13.433521 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:27:13.433521 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 137/278] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:27:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:13.438693 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77021d16-7a30-4b43-b1fe-8e8ac3489e60 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:13.513436 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:13.526309 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be958a34-6d53-40c3-9e9d-349155a70d8b None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:13.552440 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77021d16-7a30-4b43-b1fe-8e8ac3489e60 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:13.562224 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5756b379-c276-49d8-9f58-0c4a542b271e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:13.563021 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756b379-c276-49d8-9f58-0c4a542b271e None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:13.563306 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756b379-c276-49d8-9f58-0c4a542b271e None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:13.563583 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756b379-c276-49d8-9f58-0c4a542b271e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:13.564749 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5756b379-c276-49d8-9f58-0c4a542b271e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:13.566227 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5756b379-c276-49d8-9f58-0c4a542b271e None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:13.567839 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 142/279] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:27:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:13.568582 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5756b379-c276-49d8-9f58-0c4a542b271e None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:13.595486 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b88a18-44f3-42c6-8e61-20b268f99c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:13.596854 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b88a18-44f3-42c6-8e61-20b268f99c25 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:13.597139 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b88a18-44f3-42c6-8e61-20b268f99c25 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:13.597412 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b88a18-44f3-42c6-8e61-20b268f99c25 None admin] PATH_INFO: `/v3/users/ab1161bc129c415786c0870abb1d2d27` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:13.605976 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:13.616070 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b88a18-44f3-42c6-8e61-20b268f99c25 None admin] RBAC: Authorizing `identity:get_user(user_id=ab1161bc129c415786c0870abb1d2d27)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:13.617804 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b88a18-44f3-42c6-8e61-20b268f99c25 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:13.621847 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 138/280] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Jun 24 08:27:13 2022] GET /identity/v3/users/ab1161bc129c415786c0870abb1d2d27 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:13.622640 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b88a18-44f3-42c6-8e61-20b268f99c25 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:13.682905 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5756b379-c276-49d8-9f58-0c4a542b271e None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:13.720940 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520c6af7-bf66-49a0-baba-763a580b81bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:13.721599 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c6af7-bf66-49a0-baba-763a580b81bb None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:13.721908 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c6af7-bf66-49a0-baba-763a580b81bb None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:13.722178 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c6af7-bf66-49a0-baba-763a580b81bb None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:13.722925 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520c6af7-bf66-49a0-baba-763a580b81bb None admin] RBAC: Adding query filter params (name=reader) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:13.723228 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520c6af7-bf66-49a0-baba-763a580b81bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:13.724599 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520c6af7-bf66-49a0-baba-763a580b81bb None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:13.732127 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b88a18-44f3-42c6-8e61-20b268f99c25 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:13.733270 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 143/281] 10.222.0.13 () {56 vars in 1101 bytes} [Fri Jun 24 08:27:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:13.735540 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520c6af7-bf66-49a0-baba-763a580b81bb None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:13.760869 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:13.771931 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb27826-1e9c-47e7-b4a6-776a8e35f57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:13.772599 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb27826-1e9c-47e7-b4a6-776a8e35f57b None admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:13.772882 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb27826-1e9c-47e7-b4a6-776a8e35f57b None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:13.773159 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb27826-1e9c-47e7-b4a6-776a8e35f57b None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:13.774525 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb27826-1e9c-47e7-b4a6-776a8e35f57b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:13.775939 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb27826-1e9c-47e7-b4a6-776a8e35f57b None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:13.800434 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 139/282] 10.222.0.13 () {60 vars in 1132 bytes} [Fri Jun 24 08:27:13 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:27:13.806930 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb27826-1e9c-47e7-b4a6-776a8e35f57b None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:13.862688 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:13.869070 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520c6af7-bf66-49a0-baba-763a580b81bb None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:13.911222 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e16ee0-2aa7-47e9-9cf1-e413b04c3a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:13.912250 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e16ee0-2aa7-47e9-9cf1-e413b04c3a2f None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:13.912739 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e16ee0-2aa7-47e9-9cf1-e413b04c3a2f None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:13.913266 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e16ee0-2aa7-47e9-9cf1-e413b04c3a2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:13.914470 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e16ee0-2aa7-47e9-9cf1-e413b04c3a2f None admin] RBAC: Adding query filter params (role.id=bd56838b71ec4a5781fe2df9f3f4f1ab, user.id=ab1161bc129c415786c0870abb1d2d27, scope.system=all) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:13.915258 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e16ee0-2aa7-47e9-9cf1-e413b04c3a2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:13.918016 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e16ee0-2aa7-47e9-9cf1-e413b04c3a2f None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:13.924555 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 144/283] 10.222.0.13 () {56 vars in 1308 bytes} [Fri Jun 24 08:27:13 2022] GET /identity/v3/role_assignments?user.id=ab1161bc129c415786c0870abb1d2d27&scope.system=all&role.id=bd56838b71ec4a5781fe2df9f3f4f1ab => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:13.961821 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb27826-1e9c-47e7-b4a6-776a8e35f57b None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:13.966422 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f9dba-68ab-48bf-ab94-f18be6a20a78 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:13.966822 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f9dba-68ab-48bf-ab94-f18be6a20a78 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:13.967175 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f9dba-68ab-48bf-ab94-f18be6a20a78 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:13.968852 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 145/284] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:13.974650 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4d199-f233-4e56-8432-52ff3af54aad None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:13.975268 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4d199-f233-4e56-8432-52ff3af54aad None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:13.975553 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4d199-f233-4e56-8432-52ff3af54aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:13.998208 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-90f4d199-f233-4e56-8432-52ff3af54aad None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:13.999689 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-90f4d199-f233-4e56-8432-52ff3af54aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:14.007825 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e82dc3c-097d-4b0f-baf4-2bba08f109f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:14.008603 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e82dc3c-097d-4b0f-baf4-2bba08f109f8 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:14.008825 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e82dc3c-097d-4b0f-baf4-2bba08f109f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:14.009179 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e82dc3c-097d-4b0f-baf4-2bba08f109f8 None admin] PATH_INFO: `/v3/users/efc8f14f69c74fb4a15e7b37360af9c5` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:14.011230 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e82dc3c-097d-4b0f-baf4-2bba08f109f8 None admin] RBAC: Authorizing `identity:get_user(user_id=efc8f14f69c74fb4a15e7b37360af9c5)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:14.013245 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e82dc3c-097d-4b0f-baf4-2bba08f109f8 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:14.016021 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 140/285] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Jun 24 08:27:13 2022] GET /identity/v3/users/efc8f14f69c74fb4a15e7b37360af9c5 => generated 297 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:14.021602 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e82dc3c-097d-4b0f-baf4-2bba08f109f8 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:14.084876 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:14.107073 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 146/286] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 08:27:14.134383 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f4d199-f233-4e56-8432-52ff3af54aad None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:14.159537 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e82dc3c-097d-4b0f-baf4-2bba08f109f8 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:14.202524 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a0eade-290f-48f1-bf7f-c29aeae01edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:14.203384 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0eade-290f-48f1-bf7f-c29aeae01edc None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:14.203673 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0eade-290f-48f1-bf7f-c29aeae01edc None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:14.203951 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0eade-290f-48f1-bf7f-c29aeae01edc None admin] PATH_INFO: `/v3/projects/58dfc569119141c688e8c2f5c4faab17` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:14.205182 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a0eade-290f-48f1-bf7f-c29aeae01edc None admin] RBAC: Authorizing `identity:get_project(project_id=58dfc569119141c688e8c2f5c4faab17)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:14.206809 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a0eade-290f-48f1-bf7f-c29aeae01edc None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:14.208659 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 141/287] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:27:14 2022] GET /identity/v3/projects/58dfc569119141c688e8c2f5c4faab17 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:14.214656 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a0eade-290f-48f1-bf7f-c29aeae01edc None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:14.262114 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:14.271003 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f4d199-f233-4e56-8432-52ff3af54aad None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:14.318049 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5650f1-62db-4d44-9f6c-54f339192ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:14.318910 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5650f1-62db-4d44-9f6c-54f339192ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:14.319201 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5650f1-62db-4d44-9f6c-54f339192ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:14.319477 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5650f1-62db-4d44-9f6c-54f339192ac7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:14.331838 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5650f1-62db-4d44-9f6c-54f339192ac7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:27:14.339630 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a0eade-290f-48f1-bf7f-c29aeae01edc None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:14.345225 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-9c5650f1-62db-4d44-9f6c-54f339192ac7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:27:14.345225 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:14.345225 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:14.345225 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:14.345225 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:14.345225 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:14.345225 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:14.345225 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:14.345225 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:14.345225 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:14.345225 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:14.345225 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:14.345225 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:27:14.345225 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:27:14.345225 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:27:14.345225 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:27:14.346148 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:27:14.346148 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:27:14.346148 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:27:14.346148 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:27:14.346148 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:14.346148 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:14.346148 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:14.346148 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:14.346148 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:14.346148 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:14.346148 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:14.346148 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:14.346148 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:14.346148 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:14.346148 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:14.346148 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:14.346924 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:14.346924 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:14.346924 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:14.346924 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:14.346924 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:27:14.346924 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:27:14.346924 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:27:14.346924 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:27:14.346924 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:27:14.346924 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:27:14.346924 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:27:14.346924 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:14.348018 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 147/288] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:27:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:27:14.354172 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5650f1-62db-4d44-9f6c-54f339192ac7 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:14.368252 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:14.408611 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3e3a27-bd46-48d4-89c3-71870a93bdd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:14.409582 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e3a27-bd46-48d4-89c3-71870a93bdd3 None admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:14.409994 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e3a27-bd46-48d4-89c3-71870a93bdd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:14.410365 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e3a27-bd46-48d4-89c3-71870a93bdd3 None admin] PATH_INFO: `/v3/projects/58dfc569119141c688e8c2f5c4faab17/users/efc8f14f69c74fb4a15e7b37360af9c5/roles/986a50f3abf14fdab55012a8e52719c7` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:14.415889 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3e3a27-bd46-48d4-89c3-71870a93bdd3 None admin] RBAC: Authorizing `identity:create_grant(project_id=58dfc569119141c688e8c2f5c4faab17, user_id=efc8f14f69c74fb4a15e7b37360af9c5, role_id=986a50f3abf14fdab55012a8e52719c7)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:14.417573 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3e3a27-bd46-48d4-89c3-71870a93bdd3 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:14.460590 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 142/289] 10.222.0.13 () {58 vars in 1439 bytes} [Fri Jun 24 08:27:14 2022] PUT /identity/v3/projects/58dfc569119141c688e8c2f5c4faab17/users/efc8f14f69c74fb4a15e7b37360af9c5/roles/986a50f3abf14fdab55012a8e52719c7 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:27:14.492140 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5650f1-62db-4d44-9f6c-54f339192ac7 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:14.540619 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e40a2d9-5480-4492-87cd-241e3775846a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:14.541287 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40a2d9-5480-4492-87cd-241e3775846a None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:14.541569 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40a2d9-5480-4492-87cd-241e3775846a None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:14.541908 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40a2d9-5480-4492-87cd-241e3775846a None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:14.542668 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e40a2d9-5480-4492-87cd-241e3775846a None admin] RBAC: Adding query filter params (name=member) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:14.542971 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e40a2d9-5480-4492-87cd-241e3775846a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:14.544633 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e40a2d9-5480-4492-87cd-241e3775846a None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:14.554572 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 148/290] 10.222.0.13 () {56 vars in 1101 bytes} [Fri Jun 24 08:27:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:14.560759 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3e3a27-bd46-48d4-89c3-71870a93bdd3 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:14.714779 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3e3a27-bd46-48d4-89c3-71870a93bdd3 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:14.752118 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b7a1c7-6339-4ec1-b51c-5330c4320c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:14.752993 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b7a1c7-6339-4ec1-b51c-5330c4320c3f None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:14.753283 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b7a1c7-6339-4ec1-b51c-5330c4320c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:14.753544 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b7a1c7-6339-4ec1-b51c-5330c4320c3f None admin] PATH_INFO: `/v3/users/eba4bb48555c48eca3b0e1f2995bac9a` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:14.754830 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b7a1c7-6339-4ec1-b51c-5330c4320c3f None admin] RBAC: Authorizing `identity:get_user(user_id=eba4bb48555c48eca3b0e1f2995bac9a)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:14.756294 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b7a1c7-6339-4ec1-b51c-5330c4320c3f None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:14.758959 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 143/291] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Jun 24 08:27:14 2022] GET /identity/v3/users/eba4bb48555c48eca3b0e1f2995bac9a => generated 297 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:14.764312 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e40a2d9-5480-4492-87cd-241e3775846a None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:14.783759 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:14.892802 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e40a2d9-5480-4492-87cd-241e3775846a None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:14.935559 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e999cf57-c34a-490a-af7b-1611a89d2f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:14.936212 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999cf57-c34a-490a-af7b-1611a89d2f27 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:14.936496 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999cf57-c34a-490a-af7b-1611a89d2f27 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:14.936772 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999cf57-c34a-490a-af7b-1611a89d2f27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:15.033275 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e999cf57-c34a-490a-af7b-1611a89d2f27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:15.034687 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e999cf57-c34a-490a-af7b-1611a89d2f27 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:15.057621 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:15.193617 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 149/292] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:27:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 08:27:15.200405 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b7a1c7-6339-4ec1-b51c-5330c4320c3f None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:15.280170 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:15.339504 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b7a1c7-6339-4ec1-b51c-5330c4320c3f None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:15.395591 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83b7d68-6bd2-4220-ae9d-47e5719dc286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:15.396454 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b7d68-6bd2-4220-ae9d-47e5719dc286 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:15.396737 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b7d68-6bd2-4220-ae9d-47e5719dc286 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:15.397014 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b7d68-6bd2-4220-ae9d-47e5719dc286 None admin] PATH_INFO: `/v3/projects/58dfc569119141c688e8c2f5c4faab17` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:15.398302 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83b7d68-6bd2-4220-ae9d-47e5719dc286 None admin] RBAC: Authorizing `identity:get_project(project_id=58dfc569119141c688e8c2f5c4faab17)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:15.400286 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83b7d68-6bd2-4220-ae9d-47e5719dc286 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:15.402380 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 144/293] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:27:15 2022] GET /identity/v3/projects/58dfc569119141c688e8c2f5c4faab17 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:15.409728 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e999cf57-c34a-490a-af7b-1611a89d2f27 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:15.533475 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e999cf57-c34a-490a-af7b-1611a89d2f27 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:15.558091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:15.570054 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad67b7af-519a-4dbf-b754-456b0199867e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:15.571682 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad67b7af-519a-4dbf-b754-456b0199867e None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:15.571965 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad67b7af-519a-4dbf-b754-456b0199867e None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:15.572239 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad67b7af-519a-4dbf-b754-456b0199867e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:15.573069 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad67b7af-519a-4dbf-b754-456b0199867e None admin] RBAC: Adding query filter params (role.id=bd56838b71ec4a5781fe2df9f3f4f1ab, scope.project.id=58dfc569119141c688e8c2f5c4faab17, user.id=eba4bb48555c48eca3b0e1f2995bac9a) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:15.573753 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad67b7af-519a-4dbf-b754-456b0199867e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:15.575550 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad67b7af-519a-4dbf-b754-456b0199867e None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:15.582244 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 150/294] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Jun 24 08:27:15 2022] GET /identity/v3/role_assignments?user.id=eba4bb48555c48eca3b0e1f2995bac9a&scope.project.id=58dfc569119141c688e8c2f5c4faab17&role.id=bd56838b71ec4a5781fe2df9f3f4f1ab => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:16.433061 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83abdc-26e1-46eb-a831-65cea949183b None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:16.433799 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83abdc-26e1-46eb-a831-65cea949183b None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:16.433799 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83abdc-26e1-46eb-a831-65cea949183b None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:16.435768 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 145/295] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:16.443824 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb6d98-e390-4516-bfa0-c87c96bf360d None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:16.443824 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb6d98-e390-4516-bfa0-c87c96bf360d None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:16.443824 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb6d98-e390-4516-bfa0-c87c96bf360d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:16.460477 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-aefb6d98-e390-4516-bfa0-c87c96bf360d None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:16.462338 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-aefb6d98-e390-4516-bfa0-c87c96bf360d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:16.536692 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dffef30-8c70-4ef8-b7ea-cab0b7b3d96d None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:16.536904 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dffef30-8c70-4ef8-b7ea-cab0b7b3d96d None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:16.537106 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 151/296] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 24 08:27:16.537461 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dffef30-8c70-4ef8-b7ea-cab0b7b3d96d None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:16.541710 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 146/297] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:16.544578 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea19a8-84b5-4ffe-9869-231d79a88b4e None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:16.544952 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea19a8-84b5-4ffe-9869-231d79a88b4e None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:16.545306 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea19a8-84b5-4ffe-9869-231d79a88b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:16.563195 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dffef30-8c70-4ef8-b7ea-cab0b7b3d96d None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:16.564860 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-f5ea19a8-84b5-4ffe-9869-231d79a88b4e None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:16.566103 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-f5ea19a8-84b5-4ffe-9869-231d79a88b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:16.643324 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 152/298] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:27:16.644588 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39af097-f3d2-4bc5-ba17-c785e0652ec5 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:16.644876 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39af097-f3d2-4bc5-ba17-c785e0652ec5 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:16.645148 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39af097-f3d2-4bc5-ba17-c785e0652ec5 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:16.646434 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 153/299] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:16.654764 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26057b3-045c-4d1b-85e7-97afe9971b83 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:16.655171 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26057b3-045c-4d1b-85e7-97afe9971b83 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:16.655655 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26057b3-045c-4d1b-85e7-97afe9971b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:16.674634 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-f26057b3-045c-4d1b-85e7-97afe9971b83 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:16.675815 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-f26057b3-045c-4d1b-85e7-97afe9971b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:16.676027 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dffef30-8c70-4ef8-b7ea-cab0b7b3d96d None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:16.691619 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:16.710111 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:16.716351 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0805d459-f6be-4f16-95e6-a1d29c77e4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:16.717970 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0805d459-f6be-4f16-95e6-a1d29c77e4b9 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:16.718273 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0805d459-f6be-4f16-95e6-a1d29c77e4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:16.718553 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0805d459-f6be-4f16-95e6-a1d29c77e4b9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:16.729611 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0805d459-f6be-4f16-95e6-a1d29c77e4b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:27:16.737319 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-0805d459-f6be-4f16-95e6-a1d29c77e4b9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:27:16.737319 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:16.737319 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:16.737319 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:16.737319 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:16.737319 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:16.737319 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:16.737319 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:16.737319 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:16.737319 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:16.737319 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:16.737319 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:16.737319 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:27:16.737319 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:27:16.737319 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:27:16.737319 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:27:16.738372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:27:16.738372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:27:16.738372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:27:16.738372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:27:16.738372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:16.738372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:16.738372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:16.738372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:16.738372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:16.738372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:16.738372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:16.738372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:16.738372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:16.738372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:16.738372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:16.738372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:16.739331 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:16.739331 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:16.739331 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:16.739331 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:16.739331 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:27:16.739331 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:27:16.739331 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:27:16.739331 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:27:16.739331 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:27:16.739331 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:27:16.739331 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:27:16.739331 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:27:16.746605 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 147/300] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:27:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:27:16.747569 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0805d459-f6be-4f16-95e6-a1d29c77e4b9 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:16.752581 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 154/301] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:27:16.753410 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26057b3-045c-4d1b-85e7-97afe9971b83 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:16.862733 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0805d459-f6be-4f16-95e6-a1d29c77e4b9 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:16.864763 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26057b3-045c-4d1b-85e7-97afe9971b83 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:16.897275 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2e370f-8211-48a3-a5f6-d62d7954688a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:16.898020 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e370f-8211-48a3-a5f6-d62d7954688a None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:16.898306 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e370f-8211-48a3-a5f6-d62d7954688a None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:16.898587 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e370f-8211-48a3-a5f6-d62d7954688a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:16.900273 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2e370f-8211-48a3-a5f6-d62d7954688a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:16.901348 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d2903e-d029-4e05-bd44-5190f5581a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:16.901762 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2e370f-8211-48a3-a5f6-d62d7954688a None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:16.902046 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d2903e-d029-4e05-bd44-5190f5581a20 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:16.902333 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d2903e-d029-4e05-bd44-5190f5581a20 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:16.902613 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d2903e-d029-4e05-bd44-5190f5581a20 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:16.903374 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d2903e-d029-4e05-bd44-5190f5581a20 None admin] RBAC: Adding query filter params (name=member) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:16.903496 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 148/302] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:27:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:16.903679 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d2903e-d029-4e05-bd44-5190f5581a20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:16.904126 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2e370f-8211-48a3-a5f6-d62d7954688a None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:16.905055 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d2903e-d029-4e05-bd44-5190f5581a20 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:16.915384 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 155/303] 10.222.0.13 () {56 vars in 1101 bytes} [Fri Jun 24 08:27:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:16.916635 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283aa42-07b0-4a8b-899f-4a61e1571ddc None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:16.916927 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283aa42-07b0-4a8b-899f-4a61e1571ddc None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:16.917207 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283aa42-07b0-4a8b-899f-4a61e1571ddc None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:16.921733 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 156/304] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:16.921733 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f283aa42-07b0-4a8b-899f-4a61e1571ddc None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:16.950945 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:16.952100 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:17.009652 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2e370f-8211-48a3-a5f6-d62d7954688a None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:17.023196 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f283aa42-07b0-4a8b-899f-4a61e1571ddc None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:17.043075 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30e0456-ff17-4496-8ca0-8c62ef854a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:17.043866 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e0456-ff17-4496-8ca0-8c62ef854a1b None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:17.044145 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e0456-ff17-4496-8ca0-8c62ef854a1b None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:17.044415 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e0456-ff17-4496-8ca0-8c62ef854a1b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:17.054379 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30e0456-ff17-4496-8ca0-8c62ef854a1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:27:17.057026 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca1705-1bc3-4bac-a3f5-c2a5c8e06eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:17.057684 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca1705-1bc3-4bac-a3f5-c2a5c8e06eb9 None admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:17.057998 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca1705-1bc3-4bac-a3f5-c2a5c8e06eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:17.058275 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca1705-1bc3-4bac-a3f5-c2a5c8e06eb9 None admin] PATH_INFO: `/v3/groups` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:17.058953 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ca1705-1bc3-4bac-a3f5-c2a5c8e06eb9 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:17.060379 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ca1705-1bc3-4bac-a3f5-c2a5c8e06eb9 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:17.061960 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-d30e0456-ff17-4496-8ca0-8c62ef854a1b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 08:27:17.061960 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:17.061960 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:17.061960 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:17.061960 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:17.061960 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:17.061960 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:17.061960 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:17.061960 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:17.061960 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:17.061960 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:17.061960 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:17.061960 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:27:17.061960 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:27:17.061960 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:27:17.061960 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:27:17.062921 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:27:17.062921 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:27:17.062921 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:27:17.062921 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:27:17.062921 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:17.062921 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:17.062921 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:17.062921 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:17.062921 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:17.062921 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:17.062921 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:17.062921 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:17.062921 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:17.062921 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:17.062921 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:17.062921 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:17.063779 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:17.063779 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:17.063779 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:17.063779 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:17.063779 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:27:17.063779 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:27:17.063779 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:27:17.063779 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:27:17.063779 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:27:17.063779 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:27:17.063779 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 08:27:17.063779 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:27:17.063779 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 149/305] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:27:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:17.064438 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30e0456-ff17-4496-8ca0-8c62ef854a1b None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:17.078340 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 157/306] 10.222.0.13 () {60 vars in 1134 bytes} [Fri Jun 24 08:27:16 2022] POST /identity/v3/groups => generated 229 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:27:17.079576 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66445cb-a02a-4d7a-b7a6-ee232ea40cd4 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:17.079862 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66445cb-a02a-4d7a-b7a6-ee232ea40cd4 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:17.080140 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66445cb-a02a-4d7a-b7a6-ee232ea40cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:17.098051 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-a66445cb-a02a-4d7a-b7a6-ee232ea40cd4 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:17.099298 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-a66445cb-a02a-4d7a-b7a6-ee232ea40cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:17.177428 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 158/307] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:27:17.181750 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66445cb-a02a-4d7a-b7a6-ee232ea40cd4 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:17.208119 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30e0456-ff17-4496-8ca0-8c62ef854a1b None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:17.209933 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:17.219802 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:17.252666 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b578ad-1240-4abc-af5b-d1df4f0277f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:17.253540 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b578ad-1240-4abc-af5b-d1df4f0277f5 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:17.253864 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b578ad-1240-4abc-af5b-d1df4f0277f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:17.254148 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b578ad-1240-4abc-af5b-d1df4f0277f5 None admin] PATH_INFO: `/v3/users/ab1161bc129c415786c0870abb1d2d27` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:17.255445 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b578ad-1240-4abc-af5b-d1df4f0277f5 None admin] RBAC: Authorizing `identity:get_user(user_id=ab1161bc129c415786c0870abb1d2d27)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:17.256907 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b578ad-1240-4abc-af5b-d1df4f0277f5 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:17.259734 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 150/308] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Jun 24 08:27:17 2022] GET /identity/v3/users/ab1161bc129c415786c0870abb1d2d27 => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:17.259953 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b578ad-1240-4abc-af5b-d1df4f0277f5 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:17.316020 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66445cb-a02a-4d7a-b7a6-ee232ea40cd4 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:17.356648 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4af955f-0d05-4e83-b158-5ef98efa472f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:17.357718 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af955f-0d05-4e83-b158-5ef98efa472f None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:17.357997 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af955f-0d05-4e83-b158-5ef98efa472f None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:17.358495 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af955f-0d05-4e83-b158-5ef98efa472f None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:17.359627 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4af955f-0d05-4e83-b158-5ef98efa472f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:17.360080 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4af955f-0d05-4e83-b158-5ef98efa472f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:17.361469 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4af955f-0d05-4e83-b158-5ef98efa472f None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:17.368001 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b578ad-1240-4abc-af5b-d1df4f0277f5 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:17.374207 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 159/309] 10.222.0.13 () {56 vars in 1101 bytes} [Fri Jun 24 08:27:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:17.378746 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4af955f-0d05-4e83-b158-5ef98efa472f None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:17.407523 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4447b452-18e6-4522-af41-a63e5254ca0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:17.408423 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4447b452-18e6-4522-af41-a63e5254ca0b None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:17.408782 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4447b452-18e6-4522-af41-a63e5254ca0b None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:17.409163 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4447b452-18e6-4522-af41-a63e5254ca0b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:17.420057 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4447b452-18e6-4522-af41-a63e5254ca0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:27:17.427873 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-4447b452-18e6-4522-af41-a63e5254ca0b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 08:27:17.427873 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:17.427873 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:17.427873 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:17.427873 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:17.427873 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:17.427873 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:17.427873 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:17.427873 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:17.427873 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:17.427873 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:17.427873 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:17.427873 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:27:17.427873 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:27:17.427873 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:27:17.427873 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:27:17.428904 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:27:17.428904 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:27:17.428904 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:27:17.428904 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:27:17.428904 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:17.428904 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:17.428904 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:17.428904 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:17.428904 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:17.428904 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:17.428904 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:17.428904 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:17.428904 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:17.428904 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:17.428904 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:17.428904 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:17.429682 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:17.429682 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:17.429682 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:17.429682 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:17.429682 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:27:17.429682 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:27:17.429682 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:27:17.429682 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:27:17.429682 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:27:17.429682 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:27:17.429682 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 08:27:17.429682 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:27:17.429682 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 151/310] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:27:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:17.431056 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4447b452-18e6-4522-af41-a63e5254ca0b None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:17.462384 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:17.478345 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:17.500598 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4af955f-0d05-4e83-b158-5ef98efa472f None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:17.538175 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4447b452-18e6-4522-af41-a63e5254ca0b None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:17.538393 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6994cd2f-e98f-461f-8a65-95f23d300e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:17.539179 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6994cd2f-e98f-461f-8a65-95f23d300e50 None admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:17.540053 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6994cd2f-e98f-461f-8a65-95f23d300e50 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:17.540526 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6994cd2f-e98f-461f-8a65-95f23d300e50 None admin] PATH_INFO: `/v3/system/users/ab1161bc129c415786c0870abb1d2d27/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:17.542263 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6994cd2f-e98f-461f-8a65-95f23d300e50 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=ab1161bc129c415786c0870abb1d2d27, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:17.543637 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6994cd2f-e98f-461f-8a65-95f23d300e50 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:17.561735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 160/311] 10.222.0.13 () {58 vars in 1334 bytes} [Fri Jun 24 08:27:17 2022] PUT /identity/v3/system/users/ab1161bc129c415786c0870abb1d2d27/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:27:17.561735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6994cd2f-e98f-461f-8a65-95f23d300e50 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:17.588747 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be971e5-fb41-4718-a43e-26e8f415b1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:17.589779 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be971e5-fb41-4718-a43e-26e8f415b1eb None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:17.590262 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be971e5-fb41-4718-a43e-26e8f415b1eb None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:17.590710 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be971e5-fb41-4718-a43e-26e8f415b1eb None admin] PATH_INFO: `/v3/users/db58b12211be4b508bedee00290aaa21` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:17.607486 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be971e5-fb41-4718-a43e-26e8f415b1eb None admin] RBAC: Authorizing `identity:get_user(user_id=db58b12211be4b508bedee00290aaa21)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:17.609407 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be971e5-fb41-4718-a43e-26e8f415b1eb None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:17.611418 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 152/312] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Jun 24 08:27:17 2022] GET /identity/v3/users/db58b12211be4b508bedee00290aaa21 => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:17.617362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be971e5-fb41-4718-a43e-26e8f415b1eb None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:17.713779 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6994cd2f-e98f-461f-8a65-95f23d300e50 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:17.725145 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:17.740132 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be971e5-fb41-4718-a43e-26e8f415b1eb None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:17.763815 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13dc6ca-b5b4-411c-a44a-08b5b2f8a62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:17.764774 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13dc6ca-b5b4-411c-a44a-08b5b2f8a62f None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:17.765062 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13dc6ca-b5b4-411c-a44a-08b5b2f8a62f None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:17.765344 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13dc6ca-b5b4-411c-a44a-08b5b2f8a62f None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:17.766316 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13dc6ca-b5b4-411c-a44a-08b5b2f8a62f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:17.766958 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13dc6ca-b5b4-411c-a44a-08b5b2f8a62f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:17.768810 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13dc6ca-b5b4-411c-a44a-08b5b2f8a62f None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:17.777633 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127718c8-cf2d-42a1-afa5-05015bc96d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:17.778432 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127718c8-cf2d-42a1-afa5-05015bc96d4d None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:17.778831 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127718c8-cf2d-42a1-afa5-05015bc96d4d None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:17.779179 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127718c8-cf2d-42a1-afa5-05015bc96d4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:17.780049 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 161/313] 10.222.0.13 () {56 vars in 1101 bytes} [Fri Jun 24 08:27:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:17.780443 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127718c8-cf2d-42a1-afa5-05015bc96d4d None admin] RBAC: Adding query filter params (role.id=986a50f3abf14fdab55012a8e52719c7, user.id=db58b12211be4b508bedee00290aaa21, scope.system=all) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:17.780770 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127718c8-cf2d-42a1-afa5-05015bc96d4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:17.782243 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127718c8-cf2d-42a1-afa5-05015bc96d4d None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:17.786509 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13dc6ca-b5b4-411c-a44a-08b5b2f8a62f None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:17.787587 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 153/314] 10.222.0.13 () {56 vars in 1308 bytes} [Fri Jun 24 08:27:17 2022] GET /identity/v3/role_assignments?user.id=db58b12211be4b508bedee00290aaa21&scope.system=all&role.id=986a50f3abf14fdab55012a8e52719c7 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:27:17.803266 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:17.901297 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13dc6ca-b5b4-411c-a44a-08b5b2f8a62f None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:17.937290 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e8909-f025-408b-9676-574e5496ff30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:17.938275 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e8909-f025-408b-9676-574e5496ff30 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:17.938611 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e8909-f025-408b-9676-574e5496ff30 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:17.938892 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e8909-f025-408b-9676-574e5496ff30 None admin] PATH_INFO: `/v3/users/efc8f14f69c74fb4a15e7b37360af9c5` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:17.940154 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64e8909-f025-408b-9676-574e5496ff30 None admin] RBAC: Authorizing `identity:get_user(user_id=efc8f14f69c74fb4a15e7b37360af9c5)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:17.941643 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64e8909-f025-408b-9676-574e5496ff30 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:17.943763 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 162/315] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Jun 24 08:27:17 2022] GET /identity/v3/users/efc8f14f69c74fb4a15e7b37360af9c5 => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:17.949863 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127718c8-cf2d-42a1-afa5-05015bc96d4d None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:18.061557 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127718c8-cf2d-42a1-afa5-05015bc96d4d None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:18.097680 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a65203-1fd3-47c0-b255-375b6b12d74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:18.098735 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a65203-1fd3-47c0-b255-375b6b12d74a None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:18.099236 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a65203-1fd3-47c0-b255-375b6b12d74a None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:18.099733 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a65203-1fd3-47c0-b255-375b6b12d74a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:18.123444 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:18.172910 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a65203-1fd3-47c0-b255-375b6b12d74a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:18.174744 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a65203-1fd3-47c0-b255-375b6b12d74a None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:18.285514 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 154/316] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:27:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:27:18.292085 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e8909-f025-408b-9676-574e5496ff30 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:18.377875 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:18.403140 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e8909-f025-408b-9676-574e5496ff30 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:18.438156 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d83ba6-aa41-46d3-a964-7be3364b80e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:18.439025 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d83ba6-aa41-46d3-a964-7be3364b80e7 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:18.439312 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d83ba6-aa41-46d3-a964-7be3364b80e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:18.439592 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d83ba6-aa41-46d3-a964-7be3364b80e7 None admin] PATH_INFO: `/v3/projects/58dfc569119141c688e8c2f5c4faab17` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:18.440787 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d83ba6-aa41-46d3-a964-7be3364b80e7 None admin] RBAC: Authorizing `identity:get_project(project_id=58dfc569119141c688e8c2f5c4faab17)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:18.442276 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d83ba6-aa41-46d3-a964-7be3364b80e7 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:18.444280 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 163/317] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:27:18 2022] GET /identity/v3/projects/58dfc569119141c688e8c2f5c4faab17 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:18.450784 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a65203-1fd3-47c0-b255-375b6b12d74a None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:18.511154 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:18.562332 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a65203-1fd3-47c0-b255-375b6b12d74a None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:18.597637 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde86f2a-c2a6-414a-8f22-5ca51cf80b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:18.598688 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde86f2a-c2a6-414a-8f22-5ca51cf80b1d None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:18.599196 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde86f2a-c2a6-414a-8f22-5ca51cf80b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:18.599658 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde86f2a-c2a6-414a-8f22-5ca51cf80b1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:18.600668 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde86f2a-c2a6-414a-8f22-5ca51cf80b1d None admin] RBAC: Adding query filter params (role.id=986a50f3abf14fdab55012a8e52719c7, scope.project.id=58dfc569119141c688e8c2f5c4faab17, user.id=efc8f14f69c74fb4a15e7b37360af9c5) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:18.601169 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde86f2a-c2a6-414a-8f22-5ca51cf80b1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:18.603373 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde86f2a-c2a6-414a-8f22-5ca51cf80b1d None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:18.610129 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 155/318] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Jun 24 08:27:18 2022] GET /identity/v3/role_assignments?user.id=efc8f14f69c74fb4a15e7b37360af9c5&scope.project.id=58dfc569119141c688e8c2f5c4faab17&role.id=986a50f3abf14fdab55012a8e52719c7 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:19.222039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe0c9be-946d-47d1-8ec4-66c69aac891f None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:19.222557 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe0c9be-946d-47d1-8ec4-66c69aac891f None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:19.222557 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe0c9be-946d-47d1-8ec4-66c69aac891f None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:19.224011 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 164/319] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:19.230227 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead536cb-4206-46f1-b2ef-6c981bfc6098 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:19.230509 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead536cb-4206-46f1-b2ef-6c981bfc6098 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:19.230782 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead536cb-4206-46f1-b2ef-6c981bfc6098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:19.248311 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-ead536cb-4206-46f1-b2ef-6c981bfc6098 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:19.249796 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-ead536cb-4206-46f1-b2ef-6c981bfc6098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:19.317478 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 156/320] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:27:19.339678 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe0c9be-946d-47d1-8ec4-66c69aac891f None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:19.450743 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe0c9be-946d-47d1-8ec4-66c69aac891f None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:19.486252 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d199a1e1-e61f-427d-87be-749e0b3e427b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:19.486964 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d199a1e1-e61f-427d-87be-749e0b3e427b None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:19.487243 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d199a1e1-e61f-427d-87be-749e0b3e427b None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:19.487510 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d199a1e1-e61f-427d-87be-749e0b3e427b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:19.488730 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d199a1e1-e61f-427d-87be-749e0b3e427b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:19.490245 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d199a1e1-e61f-427d-87be-749e0b3e427b None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:19.492009 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 165/321] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:27:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:19.498645 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead536cb-4206-46f1-b2ef-6c981bfc6098 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:19.548458 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:19.604953 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead536cb-4206-46f1-b2ef-6c981bfc6098 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:19.638642 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d2be25-5d41-464c-b85f-d54b45f8fe71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:19.639301 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d2be25-5d41-464c-b85f-d54b45f8fe71 None admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:19.639583 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d2be25-5d41-464c-b85f-d54b45f8fe71 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:19.639855 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d2be25-5d41-464c-b85f-d54b45f8fe71 None admin] PATH_INFO: `/v3/groups` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:19.640524 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d2be25-5d41-464c-b85f-d54b45f8fe71 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:19.641989 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d2be25-5d41-464c-b85f-d54b45f8fe71 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:19.669724 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 157/322] 10.222.0.13 () {60 vars in 1134 bytes} [Fri Jun 24 08:27:19 2022] POST /identity/v3/groups => generated 226 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:27:19.722567 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef143b-03da-42ac-bce1-69106e742c23 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:19.722858 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef143b-03da-42ac-bce1-69106e742c23 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:19.723277 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef143b-03da-42ac-bce1-69106e742c23 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:19.724801 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 166/323] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:19.731695 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eee3ad-99e1-4820-aa5d-33cfffe45dc1 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:19.732026 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eee3ad-99e1-4820-aa5d-33cfffe45dc1 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:19.732258 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eee3ad-99e1-4820-aa5d-33cfffe45dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:19.751372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-e3eee3ad-99e1-4820-aa5d-33cfffe45dc1 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:19.752498 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-e3eee3ad-99e1-4820-aa5d-33cfffe45dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:19.831321 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 158/324] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:27:19.854799 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ef143b-03da-42ac-bce1-69106e742c23 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:19.871709 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:19.974289 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ef143b-03da-42ac-bce1-69106e742c23 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:19.990868 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e5b34-cab5-4a87-a817-2ce23cb62df5 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:19.991156 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e5b34-cab5-4a87-a817-2ce23cb62df5 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:19.991439 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e5b34-cab5-4a87-a817-2ce23cb62df5 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:19.993163 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 159/325] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:19.999675 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82afac2-efff-43aa-8564-04c44dca656c None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:19.999971 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82afac2-efff-43aa-8564-04c44dca656c None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:20.000249 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82afac2-efff-43aa-8564-04c44dca656c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:20.018118 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947b626-9bbd-415a-a1a7-6ede008c10b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:20.018473 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-c82afac2-efff-43aa-8564-04c44dca656c None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:20.018861 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947b626-9bbd-415a-a1a7-6ede008c10b8 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:20.019147 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947b626-9bbd-415a-a1a7-6ede008c10b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:20.019425 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947b626-9bbd-415a-a1a7-6ede008c10b8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:20.021316 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-c82afac2-efff-43aa-8564-04c44dca656c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:20.031761 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7947b626-9bbd-415a-a1a7-6ede008c10b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:27:20.043384 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-7947b626-9bbd-415a-a1a7-6ede008c10b8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:27:20.043384 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:20.043384 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:20.043384 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:20.043384 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:20.043384 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:20.043384 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:20.043384 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:20.043384 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:20.043384 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:20.043384 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:20.043384 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:20.043384 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:27:20.043384 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:27:20.043384 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:27:20.043384 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:27:20.044314 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:27:20.044314 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:27:20.044314 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:27:20.044314 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:27:20.044314 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:20.044314 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:20.044314 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:20.044314 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:20.044314 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:20.044314 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:20.044314 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:20.044314 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:20.044314 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:20.044314 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:20.044314 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:20.044314 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:20.045086 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:20.045086 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:20.045086 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:20.045086 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:20.045086 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:27:20.045086 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:27:20.045086 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:27:20.045086 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:27:20.045086 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:27:20.045086 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:27:20.045086 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:27:20.045086 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:20.048517 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 167/326] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:27:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:27:20.051301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947b626-9bbd-415a-a1a7-6ede008c10b8 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:20.063720 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:20.114588 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 160/327] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 08:27:20.143247 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:20.150793 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82afac2-efff-43aa-8564-04c44dca656c None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:20.188896 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947b626-9bbd-415a-a1a7-6ede008c10b8 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:20.253158 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6d3a2c-7a21-468b-b9ed-b15e023cb50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:20.253947 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d3a2c-7a21-468b-b9ed-b15e023cb50d None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:20.254193 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d3a2c-7a21-468b-b9ed-b15e023cb50d None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:20.254470 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d3a2c-7a21-468b-b9ed-b15e023cb50d None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:20.255228 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6d3a2c-7a21-468b-b9ed-b15e023cb50d None admin] RBAC: Adding query filter params (name=member) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:20.255542 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6d3a2c-7a21-468b-b9ed-b15e023cb50d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:20.258553 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6d3a2c-7a21-468b-b9ed-b15e023cb50d None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:20.269429 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 168/328] 10.222.0.13 () {56 vars in 1101 bytes} [Fri Jun 24 08:27:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:27:20.276933 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6d3a2c-7a21-468b-b9ed-b15e023cb50d None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:20.301895 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82afac2-efff-43aa-8564-04c44dca656c None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:20.342564 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b70d3-b13d-4358-b796-15596091c9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:20.343743 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b70d3-b13d-4358-b796-15596091c9c3 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:20.344918 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b70d3-b13d-4358-b796-15596091c9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:20.345403 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b70d3-b13d-4358-b796-15596091c9c3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:20.360459 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3b70d3-b13d-4358-b796-15596091c9c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:27:20.370305 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-8c3b70d3-b13d-4358-b796-15596091c9c3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 08:27:20.370305 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:20.370305 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:20.370305 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:20.370305 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:20.370305 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:20.370305 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:20.370305 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:20.370305 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:20.370305 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:20.370305 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:20.370305 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:20.370305 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:27:20.370305 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:27:20.370305 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:27:20.370305 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:27:20.371209 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:27:20.371209 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:27:20.371209 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:27:20.371209 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:27:20.371209 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:20.371209 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:20.371209 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:20.371209 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:20.371209 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:20.371209 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:20.371209 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:20.371209 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:20.371209 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:20.371209 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:20.371209 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:20.371209 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:20.371982 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:20.371982 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:20.371982 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:20.371982 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:20.371982 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:27:20.371982 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:27:20.371982 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:27:20.371982 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:27:20.371982 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:27:20.371982 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:27:20.371982 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 08:27:20.371982 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:27:20.375456 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 161/329] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:27:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:27:20.385797 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b70d3-b13d-4358-b796-15596091c9c3 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:20.412832 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6d3a2c-7a21-468b-b9ed-b15e023cb50d None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:20.425598 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:20.460933 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0f7d0f-ec30-4168-ad54-5335350d32c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:20.462699 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f7d0f-ec30-4168-ad54-5335350d32c7 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:20.462813 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f7d0f-ec30-4168-ad54-5335350d32c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:20.463080 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f7d0f-ec30-4168-ad54-5335350d32c7 None admin] PATH_INFO: `/v3/users/ab1161bc129c415786c0870abb1d2d27` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:20.464314 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0f7d0f-ec30-4168-ad54-5335350d32c7 None admin] RBAC: Authorizing `identity:get_user(user_id=ab1161bc129c415786c0870abb1d2d27)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:20.465800 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0f7d0f-ec30-4168-ad54-5335350d32c7 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:20.467448 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 169/330] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Jun 24 08:27:20 2022] GET /identity/v3/users/ab1161bc129c415786c0870abb1d2d27 => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:20.473204 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0f7d0f-ec30-4168-ad54-5335350d32c7 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:20.476003 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:20.557742 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b70d3-b13d-4358-b796-15596091c9c3 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:20.612648 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0f7d0f-ec30-4168-ad54-5335350d32c7 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:20.613546 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62768008-cc54-4ded-a79a-4d000a4876ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:20.614331 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62768008-cc54-4ded-a79a-4d000a4876ca None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:20.614993 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62768008-cc54-4ded-a79a-4d000a4876ca None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:20.615486 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62768008-cc54-4ded-a79a-4d000a4876ca None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:20.617219 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62768008-cc54-4ded-a79a-4d000a4876ca None admin] RBAC: Adding query filter params (name=reader) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:20.617816 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62768008-cc54-4ded-a79a-4d000a4876ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:20.619420 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62768008-cc54-4ded-a79a-4d000a4876ca None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:20.630279 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 162/331] 10.222.0.13 () {56 vars in 1101 bytes} [Fri Jun 24 08:27:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:20.636057 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62768008-cc54-4ded-a79a-4d000a4876ca None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:20.657733 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f49f1e1-a3a9-4a7a-abe5-4220f751e88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:20.658677 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f49f1e1-a3a9-4a7a-abe5-4220f751e88e None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:20.658971 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f49f1e1-a3a9-4a7a-abe5-4220f751e88e None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:20.659249 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f49f1e1-a3a9-4a7a-abe5-4220f751e88e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:20.660088 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f49f1e1-a3a9-4a7a-abe5-4220f751e88e None admin] RBAC: Adding query filter params (role.id=bd56838b71ec4a5781fe2df9f3f4f1ab, user.id=ab1161bc129c415786c0870abb1d2d27, scope.system=all) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:20.660408 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f49f1e1-a3a9-4a7a-abe5-4220f751e88e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:20.662358 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f49f1e1-a3a9-4a7a-abe5-4220f751e88e None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:20.679622 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 170/332] 10.222.0.13 () {56 vars in 1308 bytes} [Fri Jun 24 08:27:20 2022] GET /identity/v3/role_assignments?user.id=ab1161bc129c415786c0870abb1d2d27&scope.system=all&role.id=bd56838b71ec4a5781fe2df9f3f4f1ab => generated 510 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:27:20.754518 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:20.769835 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62768008-cc54-4ded-a79a-4d000a4876ca None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:20.820126 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea6bfa8-f003-47ae-abd8-06a7c5f9c1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:20.821672 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6bfa8-f003-47ae-abd8-06a7c5f9c1b2 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:20.822184 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6bfa8-f003-47ae-abd8-06a7c5f9c1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:20.822649 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6bfa8-f003-47ae-abd8-06a7c5f9c1b2 None admin] PATH_INFO: `/v3/users/db58b12211be4b508bedee00290aaa21` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:20.824262 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea6bfa8-f003-47ae-abd8-06a7c5f9c1b2 None admin] RBAC: Authorizing `identity:get_user(user_id=db58b12211be4b508bedee00290aaa21)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:20.825991 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea6bfa8-f003-47ae-abd8-06a7c5f9c1b2 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:20.828259 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 163/333] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Jun 24 08:27:20 2022] GET /identity/v3/users/db58b12211be4b508bedee00290aaa21 => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:27:20.833557 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f49f1e1-a3a9-4a7a-abe5-4220f751e88e None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:20.928594 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:20.979719 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f49f1e1-a3a9-4a7a-abe5-4220f751e88e None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:21.019938 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64830808-f6aa-4d2b-8211-87023d162b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:21.020808 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64830808-f6aa-4d2b-8211-87023d162b66 None admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:21.021090 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64830808-f6aa-4d2b-8211-87023d162b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:21.021365 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64830808-f6aa-4d2b-8211-87023d162b66 None admin] PATH_INFO: `/v3/system/users/db58b12211be4b508bedee00290aaa21/roles/986a50f3abf14fdab55012a8e52719c7` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:21.024951 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64830808-f6aa-4d2b-8211-87023d162b66 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=db58b12211be4b508bedee00290aaa21, role_id=986a50f3abf14fdab55012a8e52719c7)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:21.026391 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64830808-f6aa-4d2b-8211-87023d162b66 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:21.043573 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 171/334] 10.222.0.13 () {58 vars in 1334 bytes} [Fri Jun 24 08:27:20 2022] PUT /identity/v3/system/users/db58b12211be4b508bedee00290aaa21/roles/986a50f3abf14fdab55012a8e52719c7 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:27:22.548042 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1dee75-e343-41af-9158-95f2fd4ef818 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:22.549094 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1dee75-e343-41af-9158-95f2fd4ef818 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:22.549572 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1dee75-e343-41af-9158-95f2fd4ef818 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:22.551174 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 164/335] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:22.567940 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8743d-4ae4-4c7d-8441-2fe7726efa87 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:22.568856 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8743d-4ae4-4c7d-8441-2fe7726efa87 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:22.569334 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8743d-4ae4-4c7d-8441-2fe7726efa87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:22.572828 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa2a523-d18f-4cb4-9673-acb1c6ade0c3 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:22.573792 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa2a523-d18f-4cb4-9673-acb1c6ade0c3 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:22.575919 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa2a523-d18f-4cb4-9673-acb1c6ade0c3 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:22.577493 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 165/336] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:22.583226 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c0383-7b3d-4643-82c6-92634156516b None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:22.583519 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c0383-7b3d-4643-82c6-92634156516b None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:22.583797 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c0383-7b3d-4643-82c6-92634156516b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:22.597183 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-0df8743d-4ae4-4c7d-8441-2fe7726efa87 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:22.598676 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-0df8743d-4ae4-4c7d-8441-2fe7726efa87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:22.606760 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-813c0383-7b3d-4643-82c6-92634156516b None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:22.607970 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-813c0383-7b3d-4643-82c6-92634156516b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:22.718265 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 172/337] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:27:22.760937 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 166/338] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:27:22.762416 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df8743d-4ae4-4c7d-8441-2fe7726efa87 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:22.847748 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:22.888888 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c0383-7b3d-4643-82c6-92634156516b None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:22.907217 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df8743d-4ae4-4c7d-8441-2fe7726efa87 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:22.956734 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ba67d2-4c12-4b73-8b94-af0324afc7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:22.957563 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba67d2-4c12-4b73-8b94-af0324afc7f1 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:22.957870 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba67d2-4c12-4b73-8b94-af0324afc7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:22.958147 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba67d2-4c12-4b73-8b94-af0324afc7f1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:22.981924 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ba67d2-4c12-4b73-8b94-af0324afc7f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:27:22.995121 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-83ba67d2-4c12-4b73-8b94-af0324afc7f1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 08:27:22.995121 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:22.995121 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:22.995121 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:22.995121 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:22.995121 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:22.995121 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:22.995121 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:22.995121 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:22.995121 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:22.995121 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:22.995121 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:22.995121 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:27:22.995121 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:27:22.995121 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:27:22.995121 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:27:22.996040 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:27:22.996040 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:27:22.996040 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:27:22.996040 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:27:22.996040 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:22.996040 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:22.996040 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:22.996040 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:22.996040 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:22.996040 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:22.996040 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:22.996040 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:22.996040 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:22.996040 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:22.996040 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:22.996040 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:22.996808 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:22.996808 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:22.996808 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:22.996808 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:22.996808 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:27:22.996808 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:27:22.996808 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:27:22.996808 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:27:22.996808 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:27:22.996808 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:27:22.996808 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 08:27:22.996808 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:22.996808 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 173/339] 10.222.0.13 () {56 vars in 1114 bytes} [Fri Jun 24 08:27:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:27:23.008732 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ba67d2-4c12-4b73-8b94-af0324afc7f1 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:23.060355 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:23.087399 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c0383-7b3d-4643-82c6-92634156516b None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:23.140143 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e379f0a-5523-4a56-97c0-f009069c1fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:23.142709 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e379f0a-5523-4a56-97c0-f009069c1fbc None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:23.143441 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e379f0a-5523-4a56-97c0-f009069c1fbc None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:23.144623 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e379f0a-5523-4a56-97c0-f009069c1fbc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:23.183105 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e379f0a-5523-4a56-97c0-f009069c1fbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:27:23.191851 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-1e379f0a-5523-4a56-97c0-f009069c1fbc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:27:23.191851 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:23.191851 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:23.191851 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:23.191851 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:23.191851 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:23.191851 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:23.191851 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:23.191851 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:23.191851 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:23.191851 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:23.191851 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:23.191851 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:27:23.191851 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:27:23.191851 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:27:23.191851 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:27:23.192807 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:27:23.192807 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:27:23.192807 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:27:23.192807 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:27:23.192807 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:23.192807 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:23.192807 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:23.192807 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:23.192807 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:23.192807 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:23.192807 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:23.192807 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:23.192807 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:23.192807 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:23.192807 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:23.192807 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:23.193581 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:23.193581 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:23.193581 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:23.193581 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:23.193581 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:27:23.193581 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:27:23.193581 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:27:23.193581 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:27:23.193581 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:27:23.193581 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:27:23.193581 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:27:23.193581 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:27:23.208525 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ba67d2-4c12-4b73-8b94-af0324afc7f1 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:23.212338 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 167/340] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:27:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:23.218713 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e379f0a-5523-4a56-97c0-f009069c1fbc None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:23.243889 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:23.294247 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d947f11-97b3-4960-a631-6821acc65773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:23.294921 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d947f11-97b3-4960-a631-6821acc65773 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:23.295204 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d947f11-97b3-4960-a631-6821acc65773 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:23.295480 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d947f11-97b3-4960-a631-6821acc65773 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:23.296233 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d947f11-97b3-4960-a631-6821acc65773 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:23.296537 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d947f11-97b3-4960-a631-6821acc65773 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:23.297942 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d947f11-97b3-4960-a631-6821acc65773 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:23.318667 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 174/341] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:27:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:27:23.322899 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106ec46-abe5-4087-a3e4-6303f815077c None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:23.323187 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106ec46-abe5-4087-a3e4-6303f815077c None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:23.323461 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106ec46-abe5-4087-a3e4-6303f815077c None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:23.328724 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 175/342] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:23 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:23.329496 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106ec46-abe5-4087-a3e4-6303f815077c None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:23.392024 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e379f0a-5523-4a56-97c0-f009069c1fbc None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:23.432815 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b6dc0b-5a3f-4f1d-914e-847fc85c0e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:23.433792 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6dc0b-5a3f-4f1d-914e-847fc85c0e49 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:23.434237 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6dc0b-5a3f-4f1d-914e-847fc85c0e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:23.434657 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6dc0b-5a3f-4f1d-914e-847fc85c0e49 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:23.435918 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b6dc0b-5a3f-4f1d-914e-847fc85c0e49 None admin] RBAC: Adding query filter params (name=member) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:23.436361 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b6dc0b-5a3f-4f1d-914e-847fc85c0e49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:23.437967 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b6dc0b-5a3f-4f1d-914e-847fc85c0e49 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:23.449370 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 168/343] 10.222.0.13 () {56 vars in 1101 bytes} [Fri Jun 24 08:27:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:23.455195 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea417fea-f3cb-4234-92f5-0c99ef12bb06 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:23.455643 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea417fea-f3cb-4234-92f5-0c99ef12bb06 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:23.456068 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea417fea-f3cb-4234-92f5-0c99ef12bb06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:23.475520 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-ea417fea-f3cb-4234-92f5-0c99ef12bb06 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:23.477664 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-ea417fea-f3cb-4234-92f5-0c99ef12bb06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:23.488374 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:23.562172 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 169/344] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:27:23.563839 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea417fea-f3cb-4234-92f5-0c99ef12bb06 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:23.633660 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106ec46-abe5-4087-a3e4-6303f815077c None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:23.682416 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2bc0fc-7ac3-48ab-8739-6e75ec896d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:23.683470 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2bc0fc-7ac3-48ab-8739-6e75ec896d32 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:23.683959 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2bc0fc-7ac3-48ab-8739-6e75ec896d32 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:23.684437 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2bc0fc-7ac3-48ab-8739-6e75ec896d32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:23.684603 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea417fea-f3cb-4234-92f5-0c99ef12bb06 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:23.744034 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aff4bc2-657a-47f9-8f1b-40fe6397b4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:23.744957 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff4bc2-657a-47f9-8f1b-40fe6397b4e2 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:23.745391 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff4bc2-657a-47f9-8f1b-40fe6397b4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:23.745865 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff4bc2-657a-47f9-8f1b-40fe6397b4e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:23.746407 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:23.769623 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:23.775306 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2bc0fc-7ac3-48ab-8739-6e75ec896d32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:23.776826 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2bc0fc-7ac3-48ab-8739-6e75ec896d32 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:23.834984 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aff4bc2-657a-47f9-8f1b-40fe6397b4e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:23.836610 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aff4bc2-657a-47f9-8f1b-40fe6397b4e2 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:23.913104 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 176/345] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:27:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 584 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:27:23.914194 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2bc0fc-7ac3-48ab-8739-6e75ec896d32 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:23.976133 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 170/346] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:27:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:27:23.978413 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c63dbb-d17e-4ba1-870d-11fd2462b996 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:23.978914 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c63dbb-d17e-4ba1-870d-11fd2462b996 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:23.979382 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c63dbb-d17e-4ba1-870d-11fd2462b996 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:23.980884 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 171/347] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:23.982784 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c63dbb-d17e-4ba1-870d-11fd2462b996 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:24.030135 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:24.036244 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:24.065291 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2bc0fc-7ac3-48ab-8739-6e75ec896d32 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:24.118646 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef7eca9-d97a-4c29-ab53-b1fcfb96a8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:24.119953 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef7eca9-d97a-4c29-ab53-b1fcfb96a8ca None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:24.120630 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef7eca9-d97a-4c29-ab53-b1fcfb96a8ca None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:24.121112 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef7eca9-d97a-4c29-ab53-b1fcfb96a8ca None admin] PATH_INFO: `/v3/roles/member` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:24.128983 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c63dbb-d17e-4ba1-870d-11fd2462b996 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:24.139235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef7eca9-d97a-4c29-ab53-b1fcfb96a8ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:27:24.150593 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-9ef7eca9-d97a-4c29-ab53-b1fcfb96a8ca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:27:24.150593 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:24.150593 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:24.150593 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:24.150593 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:24.150593 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:24.150593 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:24.150593 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:24.150593 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:24.150593 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:24.150593 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:24.150593 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:24.150593 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:27:24.150593 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:27:24.150593 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:27:24.150593 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:27:24.151597 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:27:24.151597 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:27:24.151597 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:27:24.151597 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:27:24.151597 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:24.151597 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:24.151597 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:24.151597 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:24.151597 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:24.151597 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:24.151597 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:24.151597 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:24.151597 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:24.151597 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:24.151597 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:24.151597 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:24.152430 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:24.152430 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:24.152430 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:24.152430 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:24.152430 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:27:24.152430 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:27:24.152430 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:27:24.152430 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:27:24.152430 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:27:24.152430 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:27:24.152430 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:27:24.152430 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:24.154994 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 177/348] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:27:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:24.161272 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef7eca9-d97a-4c29-ab53-b1fcfb96a8ca None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:24.170967 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e52490b-05ff-414b-bbd6-25786afd0601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:24.171836 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52490b-05ff-414b-bbd6-25786afd0601 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:24.172122 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52490b-05ff-414b-bbd6-25786afd0601 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:24.172399 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52490b-05ff-414b-bbd6-25786afd0601 None admin] PATH_INFO: `/v3/projects/0a1cff337cd248ba94724654a9527e19` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:24.173755 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e52490b-05ff-414b-bbd6-25786afd0601 None admin] RBAC: Authorizing `identity:get_project(project_id=0a1cff337cd248ba94724654a9527e19)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:24.175171 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e52490b-05ff-414b-bbd6-25786afd0601 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:24.177004 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 172/349] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:27:23 2022] GET /identity/v3/projects/0a1cff337cd248ba94724654a9527e19 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:24.178670 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2732ae-2ca2-4285-ade9-1780f685bf42 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:24.178930 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2732ae-2ca2-4285-ade9-1780f685bf42 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:24.179293 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2732ae-2ca2-4285-ade9-1780f685bf42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:24.205260 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-7f2732ae-2ca2-4285-ade9-1780f685bf42 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:24.206501 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-7f2732ae-2ca2-4285-ade9-1780f685bf42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:24.285173 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef7eca9-d97a-4c29-ab53-b1fcfb96a8ca None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:24.293854 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 173/350] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:27:24.295529 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2732ae-2ca2-4285-ade9-1780f685bf42 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:24.328755 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f92c623-5616-4264-afb6-07a6cfd00e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:24.329651 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f92c623-5616-4264-afb6-07a6cfd00e5f None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:24.330016 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f92c623-5616-4264-afb6-07a6cfd00e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:24.330293 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f92c623-5616-4264-afb6-07a6cfd00e5f None admin] PATH_INFO: `/v3/projects/0a1cff337cd248ba94724654a9527e19` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:24.331537 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f92c623-5616-4264-afb6-07a6cfd00e5f None admin] RBAC: Authorizing `identity:get_project(project_id=0a1cff337cd248ba94724654a9527e19)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:24.333071 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f92c623-5616-4264-afb6-07a6cfd00e5f None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:24.335217 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 178/351] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:27:24 2022] GET /identity/v3/projects/0a1cff337cd248ba94724654a9527e19 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:24.336102 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f92c623-5616-4264-afb6-07a6cfd00e5f None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:24.341495 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:24.358511 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:24.451342 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2732ae-2ca2-4285-ade9-1780f685bf42 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:24.474508 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f92c623-5616-4264-afb6-07a6cfd00e5f None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:24.498320 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155cdd22-c1d8-4542-8309-379ddf066c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:24.499347 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155cdd22-c1d8-4542-8309-379ddf066c92 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:24.500094 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155cdd22-c1d8-4542-8309-379ddf066c92 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:24.500560 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155cdd22-c1d8-4542-8309-379ddf066c92 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:24.501916 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155cdd22-c1d8-4542-8309-379ddf066c92 None admin] RBAC: Adding query filter params (name=member) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:24.502677 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155cdd22-c1d8-4542-8309-379ddf066c92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:24.504711 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155cdd22-c1d8-4542-8309-379ddf066c92 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:24.516361 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 174/352] 10.222.0.13 () {56 vars in 1101 bytes} [Fri Jun 24 08:27:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:24.517524 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155cdd22-c1d8-4542-8309-379ddf066c92 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:24.523066 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf483bd5-da45-4979-85f9-946379bff829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:24.524039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf483bd5-da45-4979-85f9-946379bff829 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:24.524326 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf483bd5-da45-4979-85f9-946379bff829 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:24.524603 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf483bd5-da45-4979-85f9-946379bff829 None admin] PATH_INFO: `/v3/groups/5a531630602f4ac0832798f5e5a1855a` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:24.532225 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf483bd5-da45-4979-85f9-946379bff829 None admin] RBAC: Authorizing `identity:get_group(group_id=5a531630602f4ac0832798f5e5a1855a)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:24.533704 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf483bd5-da45-4979-85f9-946379bff829 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:24.535711 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 179/353] 10.222.0.13 () {56 vars in 1180 bytes} [Fri Jun 24 08:27:24 2022] GET /identity/v3/groups/5a531630602f4ac0832798f5e5a1855a => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:24.536484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf483bd5-da45-4979-85f9-946379bff829 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:24.619850 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:24.634343 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:24.638659 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155cdd22-c1d8-4542-8309-379ddf066c92 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:24.667919 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf483bd5-da45-4979-85f9-946379bff829 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:24.680778 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b42b66-6589-4006-96c8-85c21f0dde37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:24.681890 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b42b66-6589-4006-96c8-85c21f0dde37 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:24.682181 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b42b66-6589-4006-96c8-85c21f0dde37 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:24.682460 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b42b66-6589-4006-96c8-85c21f0dde37 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:24.697766 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b42b66-6589-4006-96c8-85c21f0dde37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:27:24.707516 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f3cf3e-ccb1-4418-a804-fc23f081d41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:24.708371 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f3cf3e-ccb1-4418-a804-fc23f081d41d None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:24.708653 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f3cf3e-ccb1-4418-a804-fc23f081d41d None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:24.708926 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f3cf3e-ccb1-4418-a804-fc23f081d41d None admin] PATH_INFO: `/v3/groups/5a531630602f4ac0832798f5e5a1855a` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:24.709125 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-23b42b66-6589-4006-96c8-85c21f0dde37 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 08:27:24.709125 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:24.709125 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:24.709125 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:24.709125 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:24.709125 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:24.709125 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:24.709125 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:24.709125 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:24.709125 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:24.709125 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:24.709125 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:24.709125 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:27:24.709125 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:27:24.709125 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:27:24.709125 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:27:24.710088 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:27:24.710088 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:27:24.710088 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:27:24.710088 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:27:24.710088 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:24.710088 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:24.710088 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:24.710088 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:24.710088 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:24.710088 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:24.710088 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:24.710088 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:24.710088 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:24.710088 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:24.710088 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:24.710088 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:24.710855 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:24.710855 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:24.710855 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:24.710855 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:24.710855 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:27:24.710855 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:27:24.710855 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:27:24.710855 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:27:24.710855 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:27:24.710855 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:27:24.710855 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 08:27:24.710855 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:27:24.710855 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 175/354] 10.222.0.13 () {56 vars in 1114 bytes} [Fri Jun 24 08:27:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:24.712132 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f3cf3e-ccb1-4418-a804-fc23f081d41d None admin] RBAC: Authorizing `identity:get_group(group_id=5a531630602f4ac0832798f5e5a1855a)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:24.712202 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5105c55b-5ae8-4477-9834-e6eae95ec9b8 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:24.712941 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5105c55b-5ae8-4477-9834-e6eae95ec9b8 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:24.712941 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5105c55b-5ae8-4477-9834-e6eae95ec9b8 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:24.713093 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f3cf3e-ccb1-4418-a804-fc23f081d41d None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:24.713989 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 176/355] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:24.718121 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5105c55b-5ae8-4477-9834-e6eae95ec9b8 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:24.718626 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 180/356] 10.222.0.13 () {56 vars in 1180 bytes} [Fri Jun 24 08:27:24 2022] GET /identity/v3/groups/5a531630602f4ac0832798f5e5a1855a => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:24.718626 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f3cf3e-ccb1-4418-a804-fc23f081d41d None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:24.839415 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5105c55b-5ae8-4477-9834-e6eae95ec9b8 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:24.843313 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f3cf3e-ccb1-4418-a804-fc23f081d41d None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:24.881170 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ba8db-68be-4122-b4c6-47fc73ab3ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:24.881875 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ba8db-68be-4122-b4c6-47fc73ab3ee0 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:24.882403 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ba8db-68be-4122-b4c6-47fc73ab3ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:24.882508 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ba8db-68be-4122-b4c6-47fc73ab3ee0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:24.886225 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977abc1e-c74d-4c0b-a0bb-fe1e0ff39f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:24.887163 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977abc1e-c74d-4c0b-a0bb-fe1e0ff39f64 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:24.888166 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977abc1e-c74d-4c0b-a0bb-fe1e0ff39f64 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:24.888456 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977abc1e-c74d-4c0b-a0bb-fe1e0ff39f64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:24.889293 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977abc1e-c74d-4c0b-a0bb-fe1e0ff39f64 None admin] RBAC: Adding query filter params (group.id=5a531630602f4ac0832798f5e5a1855a, role.id=d6ada62a74a94222b864a6c283f8d9b1, scope.project.id=0a1cff337cd248ba94724654a9527e19) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:24.889601 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977abc1e-c74d-4c0b-a0bb-fe1e0ff39f64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:24.891070 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977abc1e-c74d-4c0b-a0bb-fe1e0ff39f64 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:24.894698 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:24.899716 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 181/357] 10.222.0.13 () {56 vars in 1376 bytes} [Fri Jun 24 08:27:24 2022] GET /identity/v3/role_assignments?group.id=5a531630602f4ac0832798f5e5a1855a&scope.project.id=0a1cff337cd248ba94724654a9527e19&role.id=d6ada62a74a94222b864a6c283f8d9b1 => generated 260 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:24.902602 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f122e-ee7b-42da-bbd9-678c524d6914 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:24.902891 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f122e-ee7b-42da-bbd9-678c524d6914 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:24.903171 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f122e-ee7b-42da-bbd9-678c524d6914 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:24.904583 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 182/358] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:24.905954 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9d964-88bf-4d01-b6f3-dacef12ee7f3 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:24.906243 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9d964-88bf-4d01-b6f3-dacef12ee7f3 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:24.906519 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9d964-88bf-4d01-b6f3-dacef12ee7f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:24.922559 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:24.931084 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-9df9d964-88bf-4d01-b6f3-dacef12ee7f3 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:24.932990 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-9df9d964-88bf-4d01-b6f3-dacef12ee7f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:24.986172 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70ba8db-68be-4122-b4c6-47fc73ab3ee0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:24.987682 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70ba8db-68be-4122-b4c6-47fc73ab3ee0 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:25.045182 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 183/359] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:27:25.045917 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df9d964-88bf-4d01-b6f3-dacef12ee7f3 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:25.123857 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 177/360] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:27:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:27:25.129036 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ba8db-68be-4122-b4c6-47fc73ab3ee0 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:25.198810 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df9d964-88bf-4d01-b6f3-dacef12ee7f3 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:25.199691 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:25.243585 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:25.252957 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ba8db-68be-4122-b4c6-47fc73ab3ee0 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:25.253939 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2fe4d4-41bf-4ca0-9e28-865b5dc23870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:25.254955 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2fe4d4-41bf-4ca0-9e28-865b5dc23870 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:25.256084 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2fe4d4-41bf-4ca0-9e28-865b5dc23870 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:25.256084 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2fe4d4-41bf-4ca0-9e28-865b5dc23870 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:25.256741 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2fe4d4-41bf-4ca0-9e28-865b5dc23870 None admin] RBAC: Adding query filter params (group.id=5a531630602f4ac0832798f5e5a1855a, role.id=bd56838b71ec4a5781fe2df9f3f4f1ab, scope.project.id=0a1cff337cd248ba94724654a9527e19) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:25.257151 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2fe4d4-41bf-4ca0-9e28-865b5dc23870 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:25.260319 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2fe4d4-41bf-4ca0-9e28-865b5dc23870 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:25.270946 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 184/361] 10.222.0.13 () {56 vars in 1376 bytes} [Fri Jun 24 08:27:25 2022] GET /identity/v3/role_assignments?group.id=5a531630602f4ac0832798f5e5a1855a&scope.project.id=0a1cff337cd248ba94724654a9527e19&role.id=bd56838b71ec4a5781fe2df9f3f4f1ab => generated 260 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:25.272479 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe64e8d-240e-4828-8441-87eb8893febf None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:25.272872 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe64e8d-240e-4828-8441-87eb8893febf None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:25.273243 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe64e8d-240e-4828-8441-87eb8893febf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:25.296562 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-4fe64e8d-240e-4828-8441-87eb8893febf None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:25.298296 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-4fe64e8d-240e-4828-8441-87eb8893febf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:25.304704 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1782c8d6-5c18-42ae-951f-913228febc65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:25.305670 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1782c8d6-5c18-42ae-951f-913228febc65 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:25.306134 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1782c8d6-5c18-42ae-951f-913228febc65 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:25.306556 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1782c8d6-5c18-42ae-951f-913228febc65 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:25.307491 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1782c8d6-5c18-42ae-951f-913228febc65 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:25.307983 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1782c8d6-5c18-42ae-951f-913228febc65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:25.309615 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1782c8d6-5c18-42ae-951f-913228febc65 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:25.320435 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 178/362] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:27:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:25.322138 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1782c8d6-5c18-42ae-951f-913228febc65 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:25.395715 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 185/363] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:27:25.395715 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe64e8d-240e-4828-8441-87eb8893febf None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:25.473188 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1782c8d6-5c18-42ae-951f-913228febc65 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:25.497493 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:25.514257 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8081552d-4fcf-4fba-af8c-d53ee77e1629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:25.515413 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081552d-4fcf-4fba-af8c-d53ee77e1629 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:25.515903 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081552d-4fcf-4fba-af8c-d53ee77e1629 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:25.516379 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081552d-4fcf-4fba-af8c-d53ee77e1629 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:25.528682 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe64e8d-240e-4828-8441-87eb8893febf None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:25.529783 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8081552d-4fcf-4fba-af8c-d53ee77e1629 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:27:25.537973 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-8081552d-4fcf-4fba-af8c-d53ee77e1629 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 08:27:25.537973 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:25.537973 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:25.537973 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:25.537973 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:25.537973 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:25.537973 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:25.537973 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:25.537973 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:25.537973 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:25.537973 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:25.537973 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:25.537973 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:27:25.537973 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:27:25.537973 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:27:25.537973 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:27:25.538915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:27:25.538915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:27:25.538915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:27:25.538915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:27:25.538915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:25.538915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:25.538915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:25.538915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:25.538915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:25.538915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:25.538915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:25.538915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:25.538915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:25.538915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:25.538915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:25.538915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:25.541090 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:25.541090 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:25.541090 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:25.541090 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:25.541090 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:27:25.541090 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:27:25.541090 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:27:25.541090 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:27:25.541090 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:27:25.541090 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:27:25.541090 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 08:27:25.541090 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:27:25.544035 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 179/364] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:27:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:25.545067 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8081552d-4fcf-4fba-af8c-d53ee77e1629 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:25.559835 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:25.574317 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0412e2cc-b61f-4307-b5e6-d85dc1feac1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:25.576120 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412e2cc-b61f-4307-b5e6-d85dc1feac1c None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:25.576358 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412e2cc-b61f-4307-b5e6-d85dc1feac1c None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:25.576673 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412e2cc-b61f-4307-b5e6-d85dc1feac1c None admin] PATH_INFO: `/v3/projects/58dfc569119141c688e8c2f5c4faab17` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:25.578206 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0412e2cc-b61f-4307-b5e6-d85dc1feac1c None admin] RBAC: Authorizing `identity:get_project(project_id=58dfc569119141c688e8c2f5c4faab17)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:25.579751 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0412e2cc-b61f-4307-b5e6-d85dc1feac1c None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:25.581678 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 186/365] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:27:25 2022] GET /identity/v3/projects/58dfc569119141c688e8c2f5c4faab17 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:25.582327 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0412e2cc-b61f-4307-b5e6-d85dc1feac1c None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:25.667514 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8081552d-4fcf-4fba-af8c-d53ee77e1629 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:25.704120 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0412e2cc-b61f-4307-b5e6-d85dc1feac1c None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:25.714332 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd8ab1c-ffa2-437f-9ad7-4502f39374a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:25.715307 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd8ab1c-ffa2-437f-9ad7-4502f39374a5 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:25.715792 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd8ab1c-ffa2-437f-9ad7-4502f39374a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:25.716727 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd8ab1c-ffa2-437f-9ad7-4502f39374a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:25.755150 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ae71f1-8ac1-415a-9bfb-39a012cc9ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:25.756041 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae71f1-8ac1-415a-9bfb-39a012cc9ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:25.756329 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae71f1-8ac1-415a-9bfb-39a012cc9ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:25.756602 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae71f1-8ac1-415a-9bfb-39a012cc9ce3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:25.772999 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ae71f1-8ac1-415a-9bfb-39a012cc9ce3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:27:25.783924 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:25.784328 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-68ae71f1-8ac1-415a-9bfb-39a012cc9ce3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:27:25.784328 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:25.784328 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:25.784328 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:25.784328 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:25.784328 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:25.784328 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:25.784328 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:25.784328 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:25.784328 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:25.784328 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:25.784328 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:25.784328 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:27:25.784328 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:27:25.784328 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:27:25.784328 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:27:25.785189 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:27:25.785189 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:27:25.785189 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:27:25.785189 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:27:25.785189 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:25.785189 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:25.785189 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:25.785189 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:25.785189 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:25.785189 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:25.785189 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:25.785189 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:25.785189 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:25.785189 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:25.785189 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:25.785189 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:25.786035 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:25.786035 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:25.786035 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:25.786035 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:25.786035 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:27:25.786035 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:27:25.786035 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:27:25.786035 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:27:25.786035 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:27:25.786035 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:27:25.786035 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:27:25.786035 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:25.786035 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 187/366] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Jun 24 08:27:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:25.786681 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ae71f1-8ac1-415a-9bfb-39a012cc9ce3 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:25.811074 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd8ab1c-ffa2-437f-9ad7-4502f39374a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:25.812557 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd8ab1c-ffa2-437f-9ad7-4502f39374a5 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:25.859145 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:25.916084 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ae71f1-8ac1-415a-9bfb-39a012cc9ce3 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:25.953855 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 180/367] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:27:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:27:25.955553 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd8ab1c-ffa2-437f-9ad7-4502f39374a5 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:25.960938 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd806794-9238-4cb1-985b-ceba1c91fa2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:25.961683 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd806794-9238-4cb1-985b-ceba1c91fa2e None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:25.962030 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd806794-9238-4cb1-985b-ceba1c91fa2e None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:25.962279 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd806794-9238-4cb1-985b-ceba1c91fa2e None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:25.963046 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd806794-9238-4cb1-985b-ceba1c91fa2e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:25.963361 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd806794-9238-4cb1-985b-ceba1c91fa2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:25.964768 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd806794-9238-4cb1-985b-ceba1c91fa2e None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:25.975863 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 188/368] 10.222.0.13 () {56 vars in 1101 bytes} [Fri Jun 24 08:27:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:25.976681 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd806794-9238-4cb1-985b-ceba1c91fa2e None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:26.062714 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:26.087404 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd8ab1c-ffa2-437f-9ad7-4502f39374a5 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:26.108163 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd806794-9238-4cb1-985b-ceba1c91fa2e None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:26.125954 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:26.133910 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d598e5-28ed-47a7-a87e-1c079e7345b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:26.134998 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d598e5-28ed-47a7-a87e-1c079e7345b5 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:26.135441 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d598e5-28ed-47a7-a87e-1c079e7345b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:26.135864 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d598e5-28ed-47a7-a87e-1c079e7345b5 None admin] PATH_INFO: `/v3/groups/5a531630602f4ac0832798f5e5a1855a` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:26.139699 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d598e5-28ed-47a7-a87e-1c079e7345b5 None admin] RBAC: Authorizing `identity:get_group(group_id=5a531630602f4ac0832798f5e5a1855a)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:26.141445 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d598e5-28ed-47a7-a87e-1c079e7345b5 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:26.143733 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 181/369] 10.222.0.13 () {56 vars in 1180 bytes} [Fri Jun 24 08:27:25 2022] GET /identity/v3/groups/5a531630602f4ac0832798f5e5a1855a => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:26.144883 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d598e5-28ed-47a7-a87e-1c079e7345b5 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:26.157924 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe2cf44-3183-4df5-846c-421f0543d918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:26.158624 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2cf44-3183-4df5-846c-421f0543d918 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:26.158910 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2cf44-3183-4df5-846c-421f0543d918 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:26.159185 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2cf44-3183-4df5-846c-421f0543d918 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:26.161018 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe2cf44-3183-4df5-846c-421f0543d918 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:26.161018 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe2cf44-3183-4df5-846c-421f0543d918 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:26.162615 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe2cf44-3183-4df5-846c-421f0543d918 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:26.175744 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 189/370] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:27:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:26.176393 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe2cf44-3183-4df5-846c-421f0543d918 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:26.265875 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d598e5-28ed-47a7-a87e-1c079e7345b5 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:26.296404 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe2cf44-3183-4df5-846c-421f0543d918 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:26.303156 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d393939d-9133-4af8-9e04-8d5f7426d376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:26.304295 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d393939d-9133-4af8-9e04-8d5f7426d376 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:26.304739 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d393939d-9133-4af8-9e04-8d5f7426d376 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:26.305873 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d393939d-9133-4af8-9e04-8d5f7426d376 None admin] PATH_INFO: `/v3/projects/58dfc569119141c688e8c2f5c4faab17` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:26.307366 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d393939d-9133-4af8-9e04-8d5f7426d376 None admin] RBAC: Authorizing `identity:get_project(project_id=58dfc569119141c688e8c2f5c4faab17)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:26.309081 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d393939d-9133-4af8-9e04-8d5f7426d376 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:26.311384 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 182/371] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:27:26 2022] GET /identity/v3/projects/58dfc569119141c688e8c2f5c4faab17 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:26.312418 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d393939d-9133-4af8-9e04-8d5f7426d376 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:26.333212 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:26.334175 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42974fd5-606d-4ec1-a7e0-826a6ad022a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:26.335410 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42974fd5-606d-4ec1-a7e0-826a6ad022a4 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:26.335520 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42974fd5-606d-4ec1-a7e0-826a6ad022a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:26.335723 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42974fd5-606d-4ec1-a7e0-826a6ad022a4 None admin] PATH_INFO: `/v3/users/db58b12211be4b508bedee00290aaa21` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:26.337808 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42974fd5-606d-4ec1-a7e0-826a6ad022a4 None admin] RBAC: Authorizing `identity:get_user(user_id=db58b12211be4b508bedee00290aaa21)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:26.340186 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42974fd5-606d-4ec1-a7e0-826a6ad022a4 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:26.341882 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 190/372] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Jun 24 08:27:26 2022] GET /identity/v3/users/db58b12211be4b508bedee00290aaa21 => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:26.343383 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42974fd5-606d-4ec1-a7e0-826a6ad022a4 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:26.406528 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:26.427692 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d393939d-9133-4af8-9e04-8d5f7426d376 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:26.465633 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94d4f74-6621-41b1-b6c9-e43dc57f7661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:26.466724 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94d4f74-6621-41b1-b6c9-e43dc57f7661 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:26.467248 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94d4f74-6621-41b1-b6c9-e43dc57f7661 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:26.467710 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94d4f74-6621-41b1-b6c9-e43dc57f7661 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:26.468769 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94d4f74-6621-41b1-b6c9-e43dc57f7661 None admin] RBAC: Adding query filter params (group.id=5a531630602f4ac0832798f5e5a1855a, role.id=bd56838b71ec4a5781fe2df9f3f4f1ab, scope.project.id=58dfc569119141c688e8c2f5c4faab17) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:26.469349 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94d4f74-6621-41b1-b6c9-e43dc57f7661 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:26.471136 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94d4f74-6621-41b1-b6c9-e43dc57f7661 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:26.472569 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42974fd5-606d-4ec1-a7e0-826a6ad022a4 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:26.478088 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 183/373] 10.222.0.13 () {56 vars in 1376 bytes} [Fri Jun 24 08:27:26 2022] GET /identity/v3/role_assignments?group.id=5a531630602f4ac0832798f5e5a1855a&scope.project.id=58dfc569119141c688e8c2f5c4faab17&role.id=bd56838b71ec4a5781fe2df9f3f4f1ab => generated 260 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:26.479109 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94d4f74-6621-41b1-b6c9-e43dc57f7661 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:26.513583 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6cb9a2-3314-4766-aa70-3f1da95db2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:26.514293 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6cb9a2-3314-4766-aa70-3f1da95db2ff None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:26.514581 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6cb9a2-3314-4766-aa70-3f1da95db2ff None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:26.514859 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6cb9a2-3314-4766-aa70-3f1da95db2ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:26.577886 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:26.601557 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94d4f74-6621-41b1-b6c9-e43dc57f7661 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:26.601765 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6cb9a2-3314-4766-aa70-3f1da95db2ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:26.605592 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6cb9a2-3314-4766-aa70-3f1da95db2ff None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:26.669436 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:26.675343 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916924d5-9e29-4e01-ba5f-8b351143372d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:26.676091 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916924d5-9e29-4e01-ba5f-8b351143372d None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:26.676313 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916924d5-9e29-4e01-ba5f-8b351143372d None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:26.676690 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916924d5-9e29-4e01-ba5f-8b351143372d None admin] PATH_INFO: `/v3/groups/5a531630602f4ac0832798f5e5a1855a` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:26.678065 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916924d5-9e29-4e01-ba5f-8b351143372d None admin] RBAC: Authorizing `identity:get_group(group_id=5a531630602f4ac0832798f5e5a1855a)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:26.679624 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916924d5-9e29-4e01-ba5f-8b351143372d None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:26.682827 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 184/374] 10.222.0.13 () {56 vars in 1180 bytes} [Fri Jun 24 08:27:26 2022] GET /identity/v3/groups/5a531630602f4ac0832798f5e5a1855a => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:26.683775 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916924d5-9e29-4e01-ba5f-8b351143372d None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:26.749103 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 191/375] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:27:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:27:26.751542 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6cb9a2-3314-4766-aa70-3f1da95db2ff None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:26.841185 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916924d5-9e29-4e01-ba5f-8b351143372d None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:26.887877 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6cb9a2-3314-4766-aa70-3f1da95db2ff None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:26.928978 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ff157-194d-411e-8f7e-587617fe7d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:26.930162 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ff157-194d-411e-8f7e-587617fe7d4a None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:26.930664 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ff157-194d-411e-8f7e-587617fe7d4a None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:26.931162 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ff157-194d-411e-8f7e-587617fe7d4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:26.932221 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660ff157-194d-411e-8f7e-587617fe7d4a None admin] RBAC: Adding query filter params (group.id=5a531630602f4ac0832798f5e5a1855a, role.id=d6ada62a74a94222b864a6c283f8d9b1, scope.project.id=58dfc569119141c688e8c2f5c4faab17) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:26.932747 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660ff157-194d-411e-8f7e-587617fe7d4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:26.934451 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660ff157-194d-411e-8f7e-587617fe7d4a None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:26.940912 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 192/376] 10.222.0.13 () {56 vars in 1376 bytes} [Fri Jun 24 08:27:26 2022] GET /identity/v3/role_assignments?group.id=5a531630602f4ac0832798f5e5a1855a&scope.project.id=58dfc569119141c688e8c2f5c4faab17&role.id=d6ada62a74a94222b864a6c283f8d9b1 => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:26.942003 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ff157-194d-411e-8f7e-587617fe7d4a None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:26.942429 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a40c7cb-8396-4cff-9c85-cb25250c7843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:26.943103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40c7cb-8396-4cff-9c85-cb25250c7843 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:26.943385 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40c7cb-8396-4cff-9c85-cb25250c7843 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:26.943653 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40c7cb-8396-4cff-9c85-cb25250c7843 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:26.944505 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a40c7cb-8396-4cff-9c85-cb25250c7843 None admin] RBAC: Adding query filter params (role.id=986a50f3abf14fdab55012a8e52719c7, user.id=db58b12211be4b508bedee00290aaa21, scope.system=all) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:26.944818 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a40c7cb-8396-4cff-9c85-cb25250c7843 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:26.947423 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a40c7cb-8396-4cff-9c85-cb25250c7843 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:26.958653 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:26.978068 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 185/377] 10.222.0.13 () {56 vars in 1308 bytes} [Fri Jun 24 08:27:26 2022] GET /identity/v3/role_assignments?user.id=db58b12211be4b508bedee00290aaa21&scope.system=all&role.id=986a50f3abf14fdab55012a8e52719c7 => generated 510 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:27.074489 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ff157-194d-411e-8f7e-587617fe7d4a None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:27.114686 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10918cb-2510-4071-a461-2e717b084be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:27.115958 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10918cb-2510-4071-a461-2e717b084be2 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:27.116448 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10918cb-2510-4071-a461-2e717b084be2 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:27.116934 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10918cb-2510-4071-a461-2e717b084be2 None admin] PATH_INFO: `/v3/projects/985b4af5ece540be8554aecad33b1dc4` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:27.118524 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10918cb-2510-4071-a461-2e717b084be2 None admin] RBAC: Authorizing `identity:get_project(project_id=985b4af5ece540be8554aecad33b1dc4)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:27.120283 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10918cb-2510-4071-a461-2e717b084be2 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:27.122823 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 193/378] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:27:26 2022] GET /identity/v3/projects/985b4af5ece540be8554aecad33b1dc4 => generated 344 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:27.128173 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a40c7cb-8396-4cff-9c85-cb25250c7843 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:27.150941 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:27.267006 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a40c7cb-8396-4cff-9c85-cb25250c7843 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:27.308536 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774fa240-2c25-4ca3-8de0-561becc87f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:27.309401 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fa240-2c25-4ca3-8de0-561becc87f42 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:27.309683 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fa240-2c25-4ca3-8de0-561becc87f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:27.309978 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fa240-2c25-4ca3-8de0-561becc87f42 None admin] PATH_INFO: `/v3/groups/0d3c104b8b6442238675ca64834ba5ed` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:27.319587 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774fa240-2c25-4ca3-8de0-561becc87f42 None admin] RBAC: Authorizing `identity:get_group(group_id=0d3c104b8b6442238675ca64834ba5ed)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:27.321657 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774fa240-2c25-4ca3-8de0-561becc87f42 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:27.323769 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 186/379] 10.222.0.13 () {56 vars in 1180 bytes} [Fri Jun 24 08:27:27 2022] GET /identity/v3/groups/0d3c104b8b6442238675ca64834ba5ed => generated 229 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:27.330331 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10918cb-2510-4071-a461-2e717b084be2 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:27.371423 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad3ba55-48e1-46c2-8fa2-0b190062de6c None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:27.373938 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad3ba55-48e1-46c2-8fa2-0b190062de6c None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:27.373938 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad3ba55-48e1-46c2-8fa2-0b190062de6c None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:27.375456 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 187/380] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:27.383659 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7f5ac6-4e35-49a3-8b41-455352ad91bd None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:27.383659 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7f5ac6-4e35-49a3-8b41-455352ad91bd None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:27.383659 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7f5ac6-4e35-49a3-8b41-455352ad91bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:27.407449 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-5b7f5ac6-4e35-49a3-8b41-455352ad91bd None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:27.408629 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-5b7f5ac6-4e35-49a3-8b41-455352ad91bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:27.459201 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:27.495687 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10918cb-2510-4071-a461-2e717b084be2 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:27.501613 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:27.511850 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 188/381] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:27:27.544855 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f05317b-77d2-45ee-bd8a-398b60e2e73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:27.545514 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f05317b-77d2-45ee-bd8a-398b60e2e73c None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:27.545874 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f05317b-77d2-45ee-bd8a-398b60e2e73c None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:27.546153 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f05317b-77d2-45ee-bd8a-398b60e2e73c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:27.546983 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f05317b-77d2-45ee-bd8a-398b60e2e73c None admin] RBAC: Adding query filter params (group.id=0d3c104b8b6442238675ca64834ba5ed, role.id=acfe707eee6944ad94d1915fc9ee7ea3, scope.project.id=985b4af5ece540be8554aecad33b1dc4) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:27.547294 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f05317b-77d2-45ee-bd8a-398b60e2e73c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:27.548724 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f05317b-77d2-45ee-bd8a-398b60e2e73c None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:27.558575 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 194/382] 10.222.0.13 () {56 vars in 1376 bytes} [Fri Jun 24 08:27:27 2022] GET /identity/v3/role_assignments?group.id=0d3c104b8b6442238675ca64834ba5ed&scope.project.id=985b4af5ece540be8554aecad33b1dc4&role.id=acfe707eee6944ad94d1915fc9ee7ea3 => generated 260 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:27:27.575402 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7f5ac6-4e35-49a3-8b41-455352ad91bd None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:27.708978 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7f5ac6-4e35-49a3-8b41-455352ad91bd None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:27.754903 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f809c2d3-2f3b-49f0-a6d6-a21c143e5353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:27.755724 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809c2d3-2f3b-49f0-a6d6-a21c143e5353 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:27.756007 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809c2d3-2f3b-49f0-a6d6-a21c143e5353 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:27.756280 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809c2d3-2f3b-49f0-a6d6-a21c143e5353 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:27.775444 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f809c2d3-2f3b-49f0-a6d6-a21c143e5353 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:27:27.783524 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-f809c2d3-2f3b-49f0-a6d6-a21c143e5353 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 08:27:27.783524 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:27.783524 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:27.783524 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:27.783524 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:27.783524 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:27.783524 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:27.783524 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:27.783524 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:27.783524 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:27.783524 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:27.783524 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:27.783524 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:27:27.783524 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:27:27.783524 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:27:27.783524 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:27:27.784370 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:27:27.784370 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:27:27.784370 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:27:27.784370 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:27:27.784370 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:27.784370 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:27.784370 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:27.784370 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:27.784370 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:27.784370 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:27.784370 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:27.784370 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:27.784370 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:27.784370 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:27.784370 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:27.784370 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:27.785145 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:27.785145 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:27.785145 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:27.785145 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:27.785145 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:27:27.785145 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:27:27.785145 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:27:27.785145 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:27:27.785145 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:27:27.785145 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:27:27.785145 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 08:27:27.785145 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:27:27.785145 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 189/383] 10.222.0.13 () {56 vars in 1114 bytes} [Fri Jun 24 08:27:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:27.790943 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f05317b-77d2-45ee-bd8a-398b60e2e73c None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:27.934518 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f05317b-77d2-45ee-bd8a-398b60e2e73c None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:27.981750 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf7ff5-7d72-4f2f-a738-4d0b193fbf08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:27.982766 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf7ff5-7d72-4f2f-a738-4d0b193fbf08 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:27.983255 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf7ff5-7d72-4f2f-a738-4d0b193fbf08 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:27.983733 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf7ff5-7d72-4f2f-a738-4d0b193fbf08 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:27.984713 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bf7ff5-7d72-4f2f-a738-4d0b193fbf08 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:27.985235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bf7ff5-7d72-4f2f-a738-4d0b193fbf08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:27.986852 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bf7ff5-7d72-4f2f-a738-4d0b193fbf08 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:28.001922 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 195/384] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:27:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:28.007089 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f809c2d3-2f3b-49f0-a6d6-a21c143e5353 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:28.017374 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce06b1e-7753-4e70-b3fe-96dc1238e2dd None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:28.018001 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce06b1e-7753-4e70-b3fe-96dc1238e2dd None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:28.018495 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce06b1e-7753-4e70-b3fe-96dc1238e2dd None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:28.020061 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 196/385] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:27:28.026017 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb70adc-e5f1-4214-b4fc-369d6b5601ad None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:28.026565 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb70adc-e5f1-4214-b4fc-369d6b5601ad None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:28.027076 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb70adc-e5f1-4214-b4fc-369d6b5601ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:28.032606 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:28.045486 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-efb70adc-e5f1-4214-b4fc-369d6b5601ad None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:28.046738 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-efb70adc-e5f1-4214-b4fc-369d6b5601ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:28.123308 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:28.170528 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 197/386] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 08:27:28.179704 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f809c2d3-2f3b-49f0-a6d6-a21c143e5353 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:28.232910 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e90993b-a722-4f60-afee-38f99342d4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:28.233710 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e90993b-a722-4f60-afee-38f99342d4f8 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:28.233999 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e90993b-a722-4f60-afee-38f99342d4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:28.234280 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e90993b-a722-4f60-afee-38f99342d4f8 None admin] PATH_INFO: `/v3/groups/5a531630602f4ac0832798f5e5a1855a` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:28.235473 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e90993b-a722-4f60-afee-38f99342d4f8 None admin] RBAC: Authorizing `identity:get_group(group_id=5a531630602f4ac0832798f5e5a1855a)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:28.236942 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e90993b-a722-4f60-afee-38f99342d4f8 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:28.238910 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 190/387] 10.222.0.13 () {56 vars in 1180 bytes} [Fri Jun 24 08:27:28 2022] GET /identity/v3/groups/5a531630602f4ac0832798f5e5a1855a => generated 226 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:28.244291 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb70adc-e5f1-4214-b4fc-369d6b5601ad None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:28.252615 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e90993b-a722-4f60-afee-38f99342d4f8 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:28.422169 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e90993b-a722-4f60-afee-38f99342d4f8 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:28.427447 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb70adc-e5f1-4214-b4fc-369d6b5601ad None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:28.463157 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:28.464433 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:28.469934 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c2e218-e6fc-4e5f-bccc-9922239018da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:28.470812 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2e218-e6fc-4e5f-bccc-9922239018da None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:28.471138 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2e218-e6fc-4e5f-bccc-9922239018da None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:28.471357 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2e218-e6fc-4e5f-bccc-9922239018da None admin] PATH_INFO: `/v3/projects/0a1cff337cd248ba94724654a9527e19` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:28.472568 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c2e218-e6fc-4e5f-bccc-9922239018da None admin] RBAC: Authorizing `identity:get_project(project_id=0a1cff337cd248ba94724654a9527e19)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:28.474040 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c2e218-e6fc-4e5f-bccc-9922239018da None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:28.476451 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 198/388] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:27:28 2022] GET /identity/v3/projects/0a1cff337cd248ba94724654a9527e19 => generated 298 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:28.478830 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6c2830-8d57-46f0-8e89-e04f738498a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:28.479636 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c2830-8d57-46f0-8e89-e04f738498a2 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:28.479916 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c2830-8d57-46f0-8e89-e04f738498a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:28.480186 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c2830-8d57-46f0-8e89-e04f738498a2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:28.490515 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c2e218-e6fc-4e5f-bccc-9922239018da None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:28.495792 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6c2830-8d57-46f0-8e89-e04f738498a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:27:28.511932 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-af6c2830-8d57-46f0-8e89-e04f738498a2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:27:28.511932 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:28.511932 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:28.511932 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:28.511932 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:28.511932 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:28.511932 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:28.511932 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:28.511932 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:28.511932 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:28.511932 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:28.511932 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:28.511932 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:27:28.511932 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:27:28.511932 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:27:28.511932 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:27:28.512860 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:27:28.512860 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:27:28.512860 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:27:28.512860 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:27:28.512860 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:28.512860 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:28.512860 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:28.512860 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:28.512860 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:28.512860 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:28.512860 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:28.512860 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:28.512860 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:28.512860 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:28.512860 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:28.512860 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:28.513637 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:28.513637 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:28.513637 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:28.513637 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:28.513637 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:27:28.513637 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:27:28.513637 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:27:28.513637 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:27:28.513637 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:27:28.513637 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:27:28.513637 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:27:28.513637 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:27:28.513637 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 191/389] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:27:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:27:28.519517 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6c2830-8d57-46f0-8e89-e04f738498a2 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:28.606446 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c2e218-e6fc-4e5f-bccc-9922239018da None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:28.646498 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd2ba30-688d-4fba-808a-747080df5bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:28.647395 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2ba30-688d-4fba-808a-747080df5bc5 None admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:28.647681 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2ba30-688d-4fba-808a-747080df5bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:28.647962 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2ba30-688d-4fba-808a-747080df5bc5 None admin] PATH_INFO: `/v3/projects/0a1cff337cd248ba94724654a9527e19/groups/5a531630602f4ac0832798f5e5a1855a/roles/d6ada62a74a94222b864a6c283f8d9b1` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:28.651008 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd2ba30-688d-4fba-808a-747080df5bc5 None admin] RBAC: Authorizing `identity:create_grant(project_id=0a1cff337cd248ba94724654a9527e19, group_id=5a531630602f4ac0832798f5e5a1855a, role_id=d6ada62a74a94222b864a6c283f8d9b1)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:28.652531 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd2ba30-688d-4fba-808a-747080df5bc5 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:28.670894 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 199/390] 10.222.0.13 () {58 vars in 1442 bytes} [Fri Jun 24 08:27:28 2022] PUT /identity/v3/projects/0a1cff337cd248ba94724654a9527e19/groups/5a531630602f4ac0832798f5e5a1855a/roles/d6ada62a74a94222b864a6c283f8d9b1 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:27:28.702279 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6c2830-8d57-46f0-8e89-e04f738498a2 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:28.758381 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21225a58-2209-4f17-aad4-dbf0e5968ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:28.759186 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21225a58-2209-4f17-aad4-dbf0e5968ea7 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:28.759353 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21225a58-2209-4f17-aad4-dbf0e5968ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:28.759507 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21225a58-2209-4f17-aad4-dbf0e5968ea7 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:28.760259 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21225a58-2209-4f17-aad4-dbf0e5968ea7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:28.760563 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21225a58-2209-4f17-aad4-dbf0e5968ea7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:28.762001 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21225a58-2209-4f17-aad4-dbf0e5968ea7 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:28.778573 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 192/391] 10.222.0.13 () {56 vars in 1101 bytes} [Fri Jun 24 08:27:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:27:28.784229 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd2ba30-688d-4fba-808a-747080df5bc5 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:28.882013 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:28.942537 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd2ba30-688d-4fba-808a-747080df5bc5 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:28.982318 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38c694d-acdc-4b77-b56a-bd203eef783c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:28.983177 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38c694d-acdc-4b77-b56a-bd203eef783c None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:28.983459 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38c694d-acdc-4b77-b56a-bd203eef783c None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:28.983736 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38c694d-acdc-4b77-b56a-bd203eef783c None admin] PATH_INFO: `/v3/groups/5a531630602f4ac0832798f5e5a1855a` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:28.984928 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38c694d-acdc-4b77-b56a-bd203eef783c None admin] RBAC: Authorizing `identity:get_group(group_id=5a531630602f4ac0832798f5e5a1855a)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:28.986418 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38c694d-acdc-4b77-b56a-bd203eef783c None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:28.988399 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 200/392] 10.222.0.13 () {56 vars in 1180 bytes} [Fri Jun 24 08:27:28 2022] GET /identity/v3/groups/5a531630602f4ac0832798f5e5a1855a => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:28.993677 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21225a58-2209-4f17-aad4-dbf0e5968ea7 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:29.054572 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:29.124313 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21225a58-2209-4f17-aad4-dbf0e5968ea7 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:29.165737 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d455fc-403d-4455-ac2d-9056e951a0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:29.166603 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d455fc-403d-4455-ac2d-9056e951a0bd None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:29.166885 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d455fc-403d-4455-ac2d-9056e951a0bd None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:29.167160 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d455fc-403d-4455-ac2d-9056e951a0bd None admin] PATH_INFO: `/v3/projects/0a1cff337cd248ba94724654a9527e19` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:29.174439 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d455fc-403d-4455-ac2d-9056e951a0bd None admin] RBAC: Authorizing `identity:get_project(project_id=0a1cff337cd248ba94724654a9527e19)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:29.175890 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d455fc-403d-4455-ac2d-9056e951a0bd None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:29.177942 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 193/393] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:27:28 2022] GET /identity/v3/projects/0a1cff337cd248ba94724654a9527e19 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:29.186758 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38c694d-acdc-4b77-b56a-bd203eef783c None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:29.338741 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38c694d-acdc-4b77-b56a-bd203eef783c None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:29.375199 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:29.390082 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24efdad4-c94c-462c-9c9e-d3dfdf88dc89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:29.391053 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24efdad4-c94c-462c-9c9e-d3dfdf88dc89 None admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:29.391341 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24efdad4-c94c-462c-9c9e-d3dfdf88dc89 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:29.391663 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24efdad4-c94c-462c-9c9e-d3dfdf88dc89 None admin] PATH_INFO: `/v3/projects/0a1cff337cd248ba94724654a9527e19/groups/5a531630602f4ac0832798f5e5a1855a/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:29.393804 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24efdad4-c94c-462c-9c9e-d3dfdf88dc89 None admin] RBAC: Authorizing `identity:create_grant(project_id=0a1cff337cd248ba94724654a9527e19, group_id=5a531630602f4ac0832798f5e5a1855a, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:29.395267 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24efdad4-c94c-462c-9c9e-d3dfdf88dc89 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:29.423827 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 201/394] 10.222.0.13 () {58 vars in 1442 bytes} [Fri Jun 24 08:27:29 2022] PUT /identity/v3/projects/0a1cff337cd248ba94724654a9527e19/groups/5a531630602f4ac0832798f5e5a1855a/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:27:29.748656 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa22e0a1-b028-4327-8c6d-45829d13abfc None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:29.749027 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa22e0a1-b028-4327-8c6d-45829d13abfc None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:29.749224 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa22e0a1-b028-4327-8c6d-45829d13abfc None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:29.751205 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 194/395] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:29.756509 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b5464c-b105-44ac-8869-c35b538d2b6e None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:29.756803 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b5464c-b105-44ac-8869-c35b538d2b6e None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:29.757088 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b5464c-b105-44ac-8869-c35b538d2b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:29.767895 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4eda3-6ca1-4613-935e-5c9aaf37feb3 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:29.768432 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4eda3-6ca1-4613-935e-5c9aaf37feb3 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:29.768518 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4eda3-6ca1-4613-935e-5c9aaf37feb3 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:29.770219 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 195/396] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:27:29.778560 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2380ff7-32e2-4160-9947-9a0bbab42395 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:29.779162 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2380ff7-32e2-4160-9947-9a0bbab42395 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:29.779664 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2380ff7-32e2-4160-9947-9a0bbab42395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:29.780688 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-b0b5464c-b105-44ac-8869-c35b538d2b6e None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:29.782045 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b0b5464c-b105-44ac-8869-c35b538d2b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:29.796920 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-c2380ff7-32e2-4160-9947-9a0bbab42395 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:29.798433 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-c2380ff7-32e2-4160-9947-9a0bbab42395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:29.872635 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 202/397] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 08:27:29.880714 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 196/398] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 08:27:29.901961 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b5464c-b105-44ac-8869-c35b538d2b6e None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:29.917605 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2380ff7-32e2-4160-9947-9a0bbab42395 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:30.013712 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b5464c-b105-44ac-8869-c35b538d2b6e None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:30.033234 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:30.034768 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:30.053814 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d02edf-9f80-47dc-9454-f9a109d179ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:30.055609 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d02edf-9f80-47dc-9454-f9a109d179ab None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:30.056119 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d02edf-9f80-47dc-9454-f9a109d179ab None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:30.056598 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d02edf-9f80-47dc-9454-f9a109d179ab None admin] PATH_INFO: `/v3/roles/member` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:30.067997 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d02edf-9f80-47dc-9454-f9a109d179ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:27:30.076371 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-65d02edf-9f80-47dc-9454-f9a109d179ab None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:27:30.076371 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:30.076371 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:30.076371 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:30.076371 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:30.076371 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:30.076371 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:30.076371 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:30.076371 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:30.076371 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:30.076371 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:30.076371 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:30.076371 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:27:30.076371 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:27:30.076371 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:27:30.076371 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:27:30.077344 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:27:30.077344 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:27:30.077344 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:27:30.077344 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:27:30.077344 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:30.077344 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:30.077344 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:30.077344 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:30.077344 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:30.077344 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:30.077344 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:30.077344 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:30.077344 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:30.077344 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:30.077344 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:30.077344 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:30.078185 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:30.078185 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:30.078185 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:30.078185 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:30.078185 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:27:30.078185 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:27:30.078185 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:27:30.078185 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:27:30.078185 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:27:30.078185 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:27:30.078185 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:27:30.078185 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:30.085723 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 203/399] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:27:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:30.087734 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d02edf-9f80-47dc-9454-f9a109d179ab None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:30.121097 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2380ff7-32e2-4160-9947-9a0bbab42395 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:30.159124 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26918d0d-a3af-4316-8368-97b593653258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:30.159935 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26918d0d-a3af-4316-8368-97b593653258 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:30.160221 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26918d0d-a3af-4316-8368-97b593653258 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:30.160508 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26918d0d-a3af-4316-8368-97b593653258 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:30.171061 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26918d0d-a3af-4316-8368-97b593653258 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:27:30.179241 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-26918d0d-a3af-4316-8368-97b593653258 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 08:27:30.179241 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:30.179241 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:30.179241 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:30.179241 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:30.179241 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:30.179241 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:30.179241 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:30.179241 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:30.179241 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:30.179241 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:30.179241 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:30.179241 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:27:30.179241 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:27:30.179241 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:27:30.179241 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:27:30.180663 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:27:30.180663 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:27:30.180663 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:27:30.180663 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:27:30.180663 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:30.180663 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:30.180663 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:30.180663 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:30.180663 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:30.180663 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:30.180663 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:30.180663 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:30.180663 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:30.180663 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:30.180663 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:30.180663 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:30.181922 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:30.181922 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:30.181922 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:30.181922 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:30.181922 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:27:30.181922 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:27:30.181922 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:27:30.181922 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:27:30.181922 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:27:30.181922 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:27:30.181922 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 08:27:30.181922 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:27:30.181922 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 197/400] 10.222.0.13 () {56 vars in 1114 bytes} [Fri Jun 24 08:27:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:27:30.186904 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25856636-dcde-4605-b449-d2068dcf7098 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:30.187200 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25856636-dcde-4605-b449-d2068dcf7098 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:30.187482 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25856636-dcde-4605-b449-d2068dcf7098 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:30.189094 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 198/401] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:30.190687 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25856636-dcde-4605-b449-d2068dcf7098 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:30.217444 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d02edf-9f80-47dc-9454-f9a109d179ab None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:30.257247 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c972ea09-1a29-4cc4-b479-da1a3c1b7dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:30.257812 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972ea09-1a29-4cc4-b479-da1a3c1b7dac None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:30.258138 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972ea09-1a29-4cc4-b479-da1a3c1b7dac None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:30.258615 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972ea09-1a29-4cc4-b479-da1a3c1b7dac None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:30.259429 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c972ea09-1a29-4cc4-b479-da1a3c1b7dac None admin] RBAC: Adding query filter params (name=member) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:30.259743 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c972ea09-1a29-4cc4-b479-da1a3c1b7dac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:30.261223 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c972ea09-1a29-4cc4-b479-da1a3c1b7dac None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:30.275578 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 204/402] 10.222.0.13 () {56 vars in 1101 bytes} [Fri Jun 24 08:27:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:27:30.275578 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33017326-2564-4183-822b-6d3bab35523c None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:30.275578 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33017326-2564-4183-822b-6d3bab35523c None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:30.275578 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33017326-2564-4183-822b-6d3bab35523c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:30.293563 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-33017326-2564-4183-822b-6d3bab35523c None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:30.294881 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-33017326-2564-4183-822b-6d3bab35523c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:30.317193 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25856636-dcde-4605-b449-d2068dcf7098 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:30.327458 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:30.355544 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:30.356920 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cb7abd-b42c-453f-a3ac-81f0e9935d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:30.357902 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb7abd-b42c-453f-a3ac-81f0e9935d82 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:30.358195 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb7abd-b42c-453f-a3ac-81f0e9935d82 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:30.358473 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb7abd-b42c-453f-a3ac-81f0e9935d82 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:30.359228 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cb7abd-b42c-453f-a3ac-81f0e9935d82 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:30.359677 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cb7abd-b42c-453f-a3ac-81f0e9935d82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:30.361726 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cb7abd-b42c-453f-a3ac-81f0e9935d82 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:30.377269 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 199/404] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:27:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:27:30.377269 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cb7abd-b42c-453f-a3ac-81f0e9935d82 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:30.378484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 205/404] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:27:30.379595 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33017326-2564-4183-822b-6d3bab35523c None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:30.505000 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33017326-2564-4183-822b-6d3bab35523c None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:30.507064 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cb7abd-b42c-453f-a3ac-81f0e9935d82 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:30.541840 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11a0bcf-0f31-4271-a562-ad64d7da4639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:30.542992 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a0bcf-0f31-4271-a562-ad64d7da4639 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:30.543587 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a0bcf-0f31-4271-a562-ad64d7da4639 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:30.544035 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a0bcf-0f31-4271-a562-ad64d7da4639 None admin] PATH_INFO: `/v3/groups/5a531630602f4ac0832798f5e5a1855a` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:30.545384 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11a0bcf-0f31-4271-a562-ad64d7da4639 None admin] RBAC: Authorizing `identity:get_group(group_id=5a531630602f4ac0832798f5e5a1855a)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:30.547023 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11a0bcf-0f31-4271-a562-ad64d7da4639 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:30.547722 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2afd738-7795-4e5c-94b0-9d3f726edcbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:30.548584 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afd738-7795-4e5c-94b0-9d3f726edcbc None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:30.548870 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afd738-7795-4e5c-94b0-9d3f726edcbc None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:30.549147 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afd738-7795-4e5c-94b0-9d3f726edcbc None admin] PATH_INFO: `/v3/groups/5a531630602f4ac0832798f5e5a1855a` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:30.550047 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 206/405] 10.222.0.13 () {56 vars in 1180 bytes} [Fri Jun 24 08:27:30 2022] GET /identity/v3/groups/5a531630602f4ac0832798f5e5a1855a => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:30.550439 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2afd738-7795-4e5c-94b0-9d3f726edcbc None admin] RBAC: Authorizing `identity:get_group(group_id=5a531630602f4ac0832798f5e5a1855a)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:30.551297 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11a0bcf-0f31-4271-a562-ad64d7da4639 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:30.552073 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2afd738-7795-4e5c-94b0-9d3f726edcbc None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:30.553723 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 200/406] 10.222.0.13 () {56 vars in 1180 bytes} [Fri Jun 24 08:27:30 2022] GET /identity/v3/groups/5a531630602f4ac0832798f5e5a1855a => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:30.558964 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2afd738-7795-4e5c-94b0-9d3f726edcbc None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:30.594221 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:30.603473 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:30.657907 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11a0bcf-0f31-4271-a562-ad64d7da4639 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:30.671101 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2afd738-7795-4e5c-94b0-9d3f726edcbc None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:30.693548 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ab826-3942-4dc1-a186-326c370fe884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:30.694699 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ab826-3942-4dc1-a186-326c370fe884 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:30.695207 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ab826-3942-4dc1-a186-326c370fe884 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:30.696268 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ab826-3942-4dc1-a186-326c370fe884 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:30.707739 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2ab826-3942-4dc1-a186-326c370fe884 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:27:30.708451 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cdf8be-42cd-4c98-a0f8-6ddbc96cf44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:30.709324 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cdf8be-42cd-4c98-a0f8-6ddbc96cf44d None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:30.709610 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cdf8be-42cd-4c98-a0f8-6ddbc96cf44d None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:30.709933 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cdf8be-42cd-4c98-a0f8-6ddbc96cf44d None admin] PATH_INFO: `/v3/projects/58dfc569119141c688e8c2f5c4faab17` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:30.711126 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cdf8be-42cd-4c98-a0f8-6ddbc96cf44d None admin] RBAC: Authorizing `identity:get_project(project_id=58dfc569119141c688e8c2f5c4faab17)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:30.712668 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cdf8be-42cd-4c98-a0f8-6ddbc96cf44d None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:30.714393 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 201/407] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:27:30 2022] GET /identity/v3/projects/58dfc569119141c688e8c2f5c4faab17 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:30.715231 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cdf8be-42cd-4c98-a0f8-6ddbc96cf44d None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:30.717890 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-9d2ab826-3942-4dc1-a186-326c370fe884 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:27:30.717890 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:30.717890 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:30.717890 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:30.717890 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:30.717890 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:30.717890 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:30.717890 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:30.717890 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:30.717890 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:30.717890 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:30.717890 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:30.717890 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:27:30.717890 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:27:30.717890 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:27:30.717890 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:27:30.718724 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:27:30.718724 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:27:30.718724 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:27:30.718724 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:27:30.718724 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:30.718724 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:30.718724 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:30.718724 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:30.718724 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:30.718724 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:30.718724 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:30.718724 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:30.718724 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:30.718724 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:30.718724 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:30.718724 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:30.719528 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:30.719528 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:30.719528 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:30.719528 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:30.719528 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:27:30.719528 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:27:30.719528 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:27:30.719528 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:27:30.719528 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:27:30.719528 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:27:30.719528 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:27:30.719528 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:30.726713 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 207/408] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Jun 24 08:27:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:30.727969 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ab826-3942-4dc1-a186-326c370fe884 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:30.825466 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cdf8be-42cd-4c98-a0f8-6ddbc96cf44d None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:30.829059 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:30.833344 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ab826-3942-4dc1-a186-326c370fe884 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:30.834162 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:30.865463 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb11683-a6e3-428b-a36c-b0cf16a1f1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:30.867116 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb11683-a6e3-428b-a36c-b0cf16a1f1b5 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:30.868230 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb11683-a6e3-428b-a36c-b0cf16a1f1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:30.868743 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb11683-a6e3-428b-a36c-b0cf16a1f1b5 None admin] PATH_INFO: `/v3/projects/58dfc569119141c688e8c2f5c4faab17` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:30.870341 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb11683-a6e3-428b-a36c-b0cf16a1f1b5 None admin] RBAC: Authorizing `identity:get_project(project_id=58dfc569119141c688e8c2f5c4faab17)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:30.870711 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0a31d8-5c78-47b0-8145-71926bfe154b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:30.871594 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0a31d8-5c78-47b0-8145-71926bfe154b None admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:30.871880 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0a31d8-5c78-47b0-8145-71926bfe154b None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:30.872160 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0a31d8-5c78-47b0-8145-71926bfe154b None admin] PATH_INFO: `/v3/projects/58dfc569119141c688e8c2f5c4faab17/groups/5a531630602f4ac0832798f5e5a1855a/roles/d6ada62a74a94222b864a6c283f8d9b1` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:30.872866 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb11683-a6e3-428b-a36c-b0cf16a1f1b5 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:30.874691 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0a31d8-5c78-47b0-8145-71926bfe154b None admin] RBAC: Authorizing `identity:create_grant(project_id=58dfc569119141c688e8c2f5c4faab17, group_id=5a531630602f4ac0832798f5e5a1855a, role_id=d6ada62a74a94222b864a6c283f8d9b1)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:30.875312 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 202/409] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:27:30 2022] GET /identity/v3/projects/58dfc569119141c688e8c2f5c4faab17 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:30.876275 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb11683-a6e3-428b-a36c-b0cf16a1f1b5 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:30.882405 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0a31d8-5c78-47b0-8145-71926bfe154b None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:30.903462 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 208/410] 10.222.0.13 () {58 vars in 1442 bytes} [Fri Jun 24 08:27:30 2022] PUT /identity/v3/projects/58dfc569119141c688e8c2f5c4faab17/groups/5a531630602f4ac0832798f5e5a1855a/roles/d6ada62a74a94222b864a6c283f8d9b1 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:27:30.905013 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0a31d8-5c78-47b0-8145-71926bfe154b None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:31.036951 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0a31d8-5c78-47b0-8145-71926bfe154b None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:31.082676 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb11683-a6e3-428b-a36c-b0cf16a1f1b5 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:31.083517 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466999a2-b965-4927-a4f3-ab3839a45c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:31.084482 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466999a2-b965-4927-a4f3-ab3839a45c67 None admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:31.085087 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466999a2-b965-4927-a4f3-ab3839a45c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:31.085434 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466999a2-b965-4927-a4f3-ab3839a45c67 None admin] PATH_INFO: `/v3/projects/58dfc569119141c688e8c2f5c4faab17/groups/5a531630602f4ac0832798f5e5a1855a/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:31.088855 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466999a2-b965-4927-a4f3-ab3839a45c67 None admin] RBAC: Authorizing `identity:create_grant(project_id=58dfc569119141c688e8c2f5c4faab17, group_id=5a531630602f4ac0832798f5e5a1855a, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:31.090404 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466999a2-b965-4927-a4f3-ab3839a45c67 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:31.107972 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 209/411] 10.222.0.13 () {58 vars in 1442 bytes} [Fri Jun 24 08:27:30 2022] PUT /identity/v3/projects/58dfc569119141c688e8c2f5c4faab17/groups/5a531630602f4ac0832798f5e5a1855a/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:27:31.127710 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f095a2-786b-44c4-a022-0b3849216431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:31.128791 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f095a2-786b-44c4-a022-0b3849216431 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:31.129246 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f095a2-786b-44c4-a022-0b3849216431 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:31.129724 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f095a2-786b-44c4-a022-0b3849216431 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:31.130702 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f095a2-786b-44c4-a022-0b3849216431 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:31.131209 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f095a2-786b-44c4-a022-0b3849216431 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:31.135883 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f095a2-786b-44c4-a022-0b3849216431 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:31.146935 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 203/412] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:27:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:31.153063 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466999a2-b965-4927-a4f3-ab3839a45c67 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:31.182140 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:31.271392 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466999a2-b965-4927-a4f3-ab3839a45c67 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:31.308473 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7648537-1943-4d8f-a347-f18a925e9fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:31.309628 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7648537-1943-4d8f-a347-f18a925e9fd5 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:31.310075 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7648537-1943-4d8f-a347-f18a925e9fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:31.310623 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7648537-1943-4d8f-a347-f18a925e9fd5 None admin] PATH_INFO: `/v3/groups/0d3c104b8b6442238675ca64834ba5ed` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:31.312094 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7648537-1943-4d8f-a347-f18a925e9fd5 None admin] RBAC: Authorizing `identity:get_group(group_id=0d3c104b8b6442238675ca64834ba5ed)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:31.313672 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7648537-1943-4d8f-a347-f18a925e9fd5 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:31.315622 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 210/413] 10.222.0.13 () {56 vars in 1180 bytes} [Fri Jun 24 08:27:31 2022] GET /identity/v3/groups/0d3c104b8b6442238675ca64834ba5ed => generated 229 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:31.321860 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f095a2-786b-44c4-a022-0b3849216431 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:31.345619 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981341c-fa20-4173-963d-cb4ab10cc7cb None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:31.345928 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981341c-fa20-4173-963d-cb4ab10cc7cb None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:31.346232 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981341c-fa20-4173-963d-cb4ab10cc7cb None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:31.347781 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 211/414] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:27:31.353542 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6156e7a5-3fcb-4e6b-8c04-668945dc9b3a None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:31.353855 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6156e7a5-3fcb-4e6b-8c04-668945dc9b3a None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:31.354120 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6156e7a5-3fcb-4e6b-8c04-668945dc9b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:31.382384 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:31.382963 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-6156e7a5-3fcb-4e6b-8c04-668945dc9b3a None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:31.385003 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-6156e7a5-3fcb-4e6b-8c04-668945dc9b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:31.468757 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f095a2-786b-44c4-a022-0b3849216431 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:31.477158 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 212/415] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:27:31.505510 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c1fa25-0cbd-4137-9867-0cf9f7b87404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:31.505769 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6156e7a5-3fcb-4e6b-8c04-668945dc9b3a None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:31.506415 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c1fa25-0cbd-4137-9867-0cf9f7b87404 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:31.506697 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c1fa25-0cbd-4137-9867-0cf9f7b87404 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:31.506972 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c1fa25-0cbd-4137-9867-0cf9f7b87404 None admin] PATH_INFO: `/v3/projects/985b4af5ece540be8554aecad33b1dc4` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:31.508352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c1fa25-0cbd-4137-9867-0cf9f7b87404 None admin] RBAC: Authorizing `identity:get_project(project_id=985b4af5ece540be8554aecad33b1dc4)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:31.509955 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c1fa25-0cbd-4137-9867-0cf9f7b87404 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:31.512041 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 204/416] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:27:31 2022] GET /identity/v3/projects/985b4af5ece540be8554aecad33b1dc4 => generated 344 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:31.517494 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c1fa25-0cbd-4137-9867-0cf9f7b87404 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:31.526413 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:31.618170 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6156e7a5-3fcb-4e6b-8c04-668945dc9b3a None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:31.620233 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c1fa25-0cbd-4137-9867-0cf9f7b87404 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:31.622613 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:31.657147 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3654bce-d752-4c89-b0ac-d00ee5bb39ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:31.658158 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3654bce-d752-4c89-b0ac-d00ee5bb39ac None admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:31.658439 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3654bce-d752-4c89-b0ac-d00ee5bb39ac None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:31.658544 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bf35fd-6f94-4ed6-a584-608d5ecb8f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:31.658738 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3654bce-d752-4c89-b0ac-d00ee5bb39ac None admin] PATH_INFO: `/v3/projects/985b4af5ece540be8554aecad33b1dc4/groups/0d3c104b8b6442238675ca64834ba5ed/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:31.659311 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf35fd-6f94-4ed6-a584-608d5ecb8f47 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:31.659595 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf35fd-6f94-4ed6-a584-608d5ecb8f47 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:31.659925 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf35fd-6f94-4ed6-a584-608d5ecb8f47 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:31.660660 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3654bce-d752-4c89-b0ac-d00ee5bb39ac None admin] RBAC: Authorizing `identity:create_grant(project_id=985b4af5ece540be8554aecad33b1dc4, group_id=0d3c104b8b6442238675ca64834ba5ed, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:31.662144 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3654bce-d752-4c89-b0ac-d00ee5bb39ac None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:31.671994 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bf35fd-6f94-4ed6-a584-608d5ecb8f47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:27:31.681091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-63bf35fd-6f94-4ed6-a584-608d5ecb8f47 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 08:27:31.681091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:31.681091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:31.681091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:31.681091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:31.681091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:31.681091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:31.681091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:31.681091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:31.681091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:31.681091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:31.681091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:31.681091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:27:31.681091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:27:31.681091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:27:31.681091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:27:31.682135 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:27:31.682135 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:27:31.682135 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:27:31.682135 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:27:31.682135 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:31.682135 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:31.682135 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:31.682135 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:31.682135 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:31.682135 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:31.682135 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:31.682135 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:31.682135 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:31.682135 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:31.682135 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:31.682135 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:31.683647 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:31.683647 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:31.683647 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:31.683647 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:31.683647 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:27:31.683647 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:27:31.683647 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:27:31.683647 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:27:31.683647 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:27:31.683647 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:27:31.683647 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 08:27:31.683647 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:31.683647 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 213/417] 10.222.0.13 () {56 vars in 1114 bytes} [Fri Jun 24 08:27:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:27:31.688564 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bf35fd-6f94-4ed6-a584-608d5ecb8f47 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:31.700486 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 205/418] 10.222.0.13 () {58 vars in 1442 bytes} [Fri Jun 24 08:27:31 2022] PUT /identity/v3/projects/985b4af5ece540be8554aecad33b1dc4/groups/0d3c104b8b6442238675ca64834ba5ed/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:27:31.780000 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:31.807260 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bf35fd-6f94-4ed6-a584-608d5ecb8f47 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:31.847972 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab664fef-2292-4293-9b68-7230b96bf852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:31.848787 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab664fef-2292-4293-9b68-7230b96bf852 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:31.849081 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab664fef-2292-4293-9b68-7230b96bf852 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:31.849362 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab664fef-2292-4293-9b68-7230b96bf852 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:31.850250 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab664fef-2292-4293-9b68-7230b96bf852 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:31.850564 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab664fef-2292-4293-9b68-7230b96bf852 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:31.852611 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab664fef-2292-4293-9b68-7230b96bf852 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:31.862514 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 214/419] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:27:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:31.868770 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3654bce-d752-4c89-b0ac-d00ee5bb39ac None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:31.945165 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e2591-facc-4c93-b7ec-5590f356233e None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:31.946679 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e2591-facc-4c93-b7ec-5590f356233e None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:31.947202 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e2591-facc-4c93-b7ec-5590f356233e None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:31.949177 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 215/420] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:31.954546 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335ebdf-46c2-4cbe-90ef-38a90a11da9e None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:31.954861 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335ebdf-46c2-4cbe-90ef-38a90a11da9e None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:31.955105 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335ebdf-46c2-4cbe-90ef-38a90a11da9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:31.973284 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-2335ebdf-46c2-4cbe-90ef-38a90a11da9e None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:31.974631 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-2335ebdf-46c2-4cbe-90ef-38a90a11da9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:32.006405 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3654bce-d752-4c89-b0ac-d00ee5bb39ac None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:32.052326 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530bfafe-4a2b-4c2d-b023-1f18e43bd619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:32.053360 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530bfafe-4a2b-4c2d-b023-1f18e43bd619 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:32.053908 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530bfafe-4a2b-4c2d-b023-1f18e43bd619 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:32.054202 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530bfafe-4a2b-4c2d-b023-1f18e43bd619 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:32.068062 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 216/421] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 08:27:32.098083 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:32.100723 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335ebdf-46c2-4cbe-90ef-38a90a11da9e None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:32.175311 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530bfafe-4a2b-4c2d-b023-1f18e43bd619 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:32.176801 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530bfafe-4a2b-4c2d-b023-1f18e43bd619 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:32.226773 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:32.292095 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 206/422] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:27:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:27:32.293710 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335ebdf-46c2-4cbe-90ef-38a90a11da9e None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:32.297469 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530bfafe-4a2b-4c2d-b023-1f18e43bd619 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:32.370422 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a43e03-7657-476d-8745-d07aca2a6da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:32.371599 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a43e03-7657-476d-8745-d07aca2a6da7 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:32.372552 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a43e03-7657-476d-8745-d07aca2a6da7 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:32.372552 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a43e03-7657-476d-8745-d07aca2a6da7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:32.374106 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:32.413451 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530bfafe-4a2b-4c2d-b023-1f18e43bd619 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:32.414257 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a43e03-7657-476d-8745-d07aca2a6da7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:27:32.422929 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-66a43e03-7657-476d-8745-d07aca2a6da7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:27:32.422929 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:32.422929 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:32.422929 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:32.422929 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:32.422929 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:32.422929 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:32.422929 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:32.422929 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:32.422929 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:32.422929 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:32.422929 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:32.422929 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:27:32.422929 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:27:32.422929 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:27:32.422929 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:27:32.423871 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:27:32.423871 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:27:32.423871 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:27:32.423871 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:27:32.423871 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:32.423871 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:32.423871 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:32.423871 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:32.423871 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:32.423871 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:32.423871 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:32.423871 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:32.423871 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:32.423871 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:32.423871 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:32.423871 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:32.424657 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:32.424657 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:32.424657 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:32.424657 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:32.424657 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:27:32.424657 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:27:32.424657 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:27:32.424657 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:27:32.424657 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:27:32.424657 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:27:32.424657 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:27:32.424657 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:32.428776 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 217/423] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:27:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:27:32.434892 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a43e03-7657-476d-8745-d07aca2a6da7 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:32.460328 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d800f2-b8e9-40c4-84a2-482e22835b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:32.462121 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d800f2-b8e9-40c4-84a2-482e22835b89 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:32.462813 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d800f2-b8e9-40c4-84a2-482e22835b89 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:32.464740 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d800f2-b8e9-40c4-84a2-482e22835b89 None admin] PATH_INFO: `/v3/projects/0a1cff337cd248ba94724654a9527e19` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:32.466442 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d800f2-b8e9-40c4-84a2-482e22835b89 None admin] RBAC: Authorizing `identity:get_project(project_id=0a1cff337cd248ba94724654a9527e19)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:32.469598 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d800f2-b8e9-40c4-84a2-482e22835b89 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:32.472928 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 207/424] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:27:32 2022] GET /identity/v3/projects/0a1cff337cd248ba94724654a9527e19 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:27:32.478639 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d800f2-b8e9-40c4-84a2-482e22835b89 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:32.630975 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a43e03-7657-476d-8745-d07aca2a6da7 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:32.670981 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d800f2-b8e9-40c4-84a2-482e22835b89 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:32.694131 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:32.700999 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4902378c-f309-44a8-96ea-e8b2ec471918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:32.701976 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4902378c-f309-44a8-96ea-e8b2ec471918 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:32.703786 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4902378c-f309-44a8-96ea-e8b2ec471918 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:32.704291 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4902378c-f309-44a8-96ea-e8b2ec471918 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:32.705509 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4902378c-f309-44a8-96ea-e8b2ec471918 None admin] RBAC: Adding query filter params (name=member) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:32.706858 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4902378c-f309-44a8-96ea-e8b2ec471918 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:32.715250 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4902378c-f309-44a8-96ea-e8b2ec471918 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:32.720827 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:32.726539 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef8298c-b366-4fc1-9b54-ca69b1b840e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:32.728578 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8298c-b366-4fc1-9b54-ca69b1b840e7 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:32.729078 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8298c-b366-4fc1-9b54-ca69b1b840e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:32.729554 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8298c-b366-4fc1-9b54-ca69b1b840e7 None admin] PATH_INFO: `/v3/groups/5a531630602f4ac0832798f5e5a1855a` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:32.730232 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 218/425] 10.222.0.13 () {56 vars in 1101 bytes} [Fri Jun 24 08:27:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:32.731503 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef8298c-b366-4fc1-9b54-ca69b1b840e7 None admin] RBAC: Authorizing `identity:get_group(group_id=5a531630602f4ac0832798f5e5a1855a)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:32.733251 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef8298c-b366-4fc1-9b54-ca69b1b840e7 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:32.735263 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4902378c-f309-44a8-96ea-e8b2ec471918 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:32.735681 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 208/426] 10.222.0.13 () {56 vars in 1180 bytes} [Fri Jun 24 08:27:32 2022] GET /identity/v3/groups/5a531630602f4ac0832798f5e5a1855a => generated 226 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:32.740861 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef8298c-b366-4fc1-9b54-ca69b1b840e7 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:32.926904 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef8298c-b366-4fc1-9b54-ca69b1b840e7 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:32.931525 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4902378c-f309-44a8-96ea-e8b2ec471918 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:32.964375 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06604ead-86b1-491f-aa8b-b9bdc11d4078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:32.965352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06604ead-86b1-491f-aa8b-b9bdc11d4078 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:32.965903 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06604ead-86b1-491f-aa8b-b9bdc11d4078 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:32.966379 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06604ead-86b1-491f-aa8b-b9bdc11d4078 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:32.967419 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06604ead-86b1-491f-aa8b-b9bdc11d4078 None admin] RBAC: Adding query filter params (group.id=5a531630602f4ac0832798f5e5a1855a, role.id=d6ada62a74a94222b864a6c283f8d9b1, scope.project.id=0a1cff337cd248ba94724654a9527e19) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:32.967949 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06604ead-86b1-491f-aa8b-b9bdc11d4078 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:32.969582 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06604ead-86b1-491f-aa8b-b9bdc11d4078 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:32.976205 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 209/427] 10.222.0.13 () {56 vars in 1376 bytes} [Fri Jun 24 08:27:32 2022] GET /identity/v3/role_assignments?group.id=5a531630602f4ac0832798f5e5a1855a&scope.project.id=0a1cff337cd248ba94724654a9527e19&role.id=d6ada62a74a94222b864a6c283f8d9b1 => generated 611 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:32.990068 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae231d-1090-4371-af67-c84ed560ba68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:32.990747 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae231d-1090-4371-af67-c84ed560ba68 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:32.991021 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae231d-1090-4371-af67-c84ed560ba68 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:32.991309 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae231d-1090-4371-af67-c84ed560ba68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:33.080018 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:33.096314 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ae231d-1090-4371-af67-c84ed560ba68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:33.097801 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ae231d-1090-4371-af67-c84ed560ba68 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:33.239047 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 219/428] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:27:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 08:27:33.245854 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06604ead-86b1-491f-aa8b-b9bdc11d4078 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:33.331242 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:33.379771 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06604ead-86b1-491f-aa8b-b9bdc11d4078 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:33.420802 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566f9b45-5958-4822-9e09-55694442f6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:33.421749 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f9b45-5958-4822-9e09-55694442f6d7 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:33.422040 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f9b45-5958-4822-9e09-55694442f6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:33.422321 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f9b45-5958-4822-9e09-55694442f6d7 None admin] PATH_INFO: `/v3/projects/0a1cff337cd248ba94724654a9527e19` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:33.423559 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566f9b45-5958-4822-9e09-55694442f6d7 None admin] RBAC: Authorizing `identity:get_project(project_id=0a1cff337cd248ba94724654a9527e19)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:33.425018 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566f9b45-5958-4822-9e09-55694442f6d7 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:33.427177 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 210/429] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:27:33 2022] GET /identity/v3/projects/0a1cff337cd248ba94724654a9527e19 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:33.434811 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae231d-1090-4371-af67-c84ed560ba68 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:33.550451 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:33.551794 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a10538-280d-4ca2-b304-fab63b46b690 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:33.552101 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a10538-280d-4ca2-b304-fab63b46b690 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:33.552439 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a10538-280d-4ca2-b304-fab63b46b690 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:33.554110 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 211/430] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:33.561031 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb73a0-674b-4403-8864-fe10f1957475 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:33.561573 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb73a0-674b-4403-8864-fe10f1957475 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:33.562099 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb73a0-674b-4403-8864-fe10f1957475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:33.564749 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae231d-1090-4371-af67-c84ed560ba68 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:33.579419 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-a6bb73a0-674b-4403-8864-fe10f1957475 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:33.581069 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-a6bb73a0-674b-4403-8864-fe10f1957475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:33.601125 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d7742-1f0d-40ff-a244-5925ec01f2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:33.602664 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d7742-1f0d-40ff-a244-5925ec01f2a5 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:33.602915 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d7742-1f0d-40ff-a244-5925ec01f2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:33.603193 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d7742-1f0d-40ff-a244-5925ec01f2a5 None admin] PATH_INFO: `/v3/groups/5a531630602f4ac0832798f5e5a1855a` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:33.604449 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091d7742-1f0d-40ff-a244-5925ec01f2a5 None admin] RBAC: Authorizing `identity:get_group(group_id=5a531630602f4ac0832798f5e5a1855a)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:33.605923 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091d7742-1f0d-40ff-a244-5925ec01f2a5 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:33.607872 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 220/431] 10.222.0.13 () {56 vars in 1180 bytes} [Fri Jun 24 08:27:33 2022] GET /identity/v3/groups/5a531630602f4ac0832798f5e5a1855a => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:33.613175 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d7742-1f0d-40ff-a244-5925ec01f2a5 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:33.673767 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 212/432] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 08:27:33.698623 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bb73a0-674b-4403-8864-fe10f1957475 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:33.736000 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d7742-1f0d-40ff-a244-5925ec01f2a5 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:33.765097 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:33.776720 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f12856-16c8-4f76-927b-28e3946387b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:33.777377 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f12856-16c8-4f76-927b-28e3946387b8 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:33.777656 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f12856-16c8-4f76-927b-28e3946387b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:33.777959 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f12856-16c8-4f76-927b-28e3946387b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:33.778791 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f12856-16c8-4f76-927b-28e3946387b8 None admin] RBAC: Adding query filter params (group.id=5a531630602f4ac0832798f5e5a1855a, role.id=bd56838b71ec4a5781fe2df9f3f4f1ab, scope.project.id=0a1cff337cd248ba94724654a9527e19) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:33.779096 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f12856-16c8-4f76-927b-28e3946387b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:33.780631 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f12856-16c8-4f76-927b-28e3946387b8 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:33.793929 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 221/433] 10.222.0.13 () {56 vars in 1376 bytes} [Fri Jun 24 08:27:33 2022] GET /identity/v3/role_assignments?group.id=5a531630602f4ac0832798f5e5a1855a&scope.project.id=0a1cff337cd248ba94724654a9527e19&role.id=bd56838b71ec4a5781fe2df9f3f4f1ab => generated 611 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:27:33.800577 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5efcfb3-8ef5-467c-a2de-eb21d259002c None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:33.800760 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5efcfb3-8ef5-467c-a2de-eb21d259002c None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:33.801039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5efcfb3-8ef5-467c-a2de-eb21d259002c None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:33.802734 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 222/434] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:27:33.811477 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43bd23-7a56-4012-8a0b-bb22256a0cd4 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:33.811782 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43bd23-7a56-4012-8a0b-bb22256a0cd4 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:33.812035 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43bd23-7a56-4012-8a0b-bb22256a0cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:33.812160 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bb73a0-674b-4403-8864-fe10f1957475 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:33.831022 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-1b43bd23-7a56-4012-8a0b-bb22256a0cd4 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:33.832231 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-1b43bd23-7a56-4012-8a0b-bb22256a0cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:33.851316 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f204634a-8b25-4c0a-bac7-cc503a937018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:33.852277 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f204634a-8b25-4c0a-bac7-cc503a937018 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:33.852549 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:33.852666 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f204634a-8b25-4c0a-bac7-cc503a937018 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:33.853182 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f204634a-8b25-4c0a-bac7-cc503a937018 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:33.865658 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f204634a-8b25-4c0a-bac7-cc503a937018 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:27:33.875409 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-f204634a-8b25-4c0a-bac7-cc503a937018 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 08:27:33.875409 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:33.875409 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:33.875409 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:33.875409 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:33.875409 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:33.875409 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:33.875409 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:33.875409 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:33.875409 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:33.875409 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:33.875409 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:33.875409 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:27:33.875409 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:27:33.875409 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:27:33.875409 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:27:33.876355 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:27:33.876355 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:27:33.876355 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:27:33.876355 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:27:33.876355 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:33.876355 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:33.876355 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:33.876355 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:33.876355 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:33.876355 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:33.876355 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:33.876355 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:33.876355 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:33.876355 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:33.876355 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:33.876355 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:33.877233 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:33.877233 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:33.877233 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:33.877233 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:33.877233 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:27:33.877233 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:27:33.877233 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:27:33.877233 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:27:33.877233 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:27:33.877233 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:27:33.877233 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 08:27:33.877233 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:27:33.877233 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 213/435] 10.222.0.13 () {56 vars in 1114 bytes} [Fri Jun 24 08:27:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:27:33.886951 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f204634a-8b25-4c0a-bac7-cc503a937018 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:33.922162 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 223/436] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 08:27:33.941433 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43bd23-7a56-4012-8a0b-bb22256a0cd4 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:34.005875 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f204634a-8b25-4c0a-bac7-cc503a937018 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:34.030342 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:34.043537 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3faf03-3eae-46e6-b10a-8098c4a61126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:34.044211 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3faf03-3eae-46e6-b10a-8098c4a61126 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:34.044499 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3faf03-3eae-46e6-b10a-8098c4a61126 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:34.044773 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3faf03-3eae-46e6-b10a-8098c4a61126 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:34.045534 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3faf03-3eae-46e6-b10a-8098c4a61126 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:34.045865 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3faf03-3eae-46e6-b10a-8098c4a61126 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:34.047243 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3faf03-3eae-46e6-b10a-8098c4a61126 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:34.056449 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 214/437] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:27:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:34.061722 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3faf03-3eae-46e6-b10a-8098c4a61126 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:34.075602 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43bd23-7a56-4012-8a0b-bb22256a0cd4 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:34.130866 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd4bfe7-bcc9-4faa-8a11-44cdb0d4c1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:34.132527 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4bfe7-bcc9-4faa-8a11-44cdb0d4c1b0 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:34.133562 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4bfe7-bcc9-4faa-8a11-44cdb0d4c1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:34.134079 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4bfe7-bcc9-4faa-8a11-44cdb0d4c1b0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:34.151320 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd4bfe7-bcc9-4faa-8a11-44cdb0d4c1b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:27:34.160240 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-5fd4bfe7-bcc9-4faa-8a11-44cdb0d4c1b0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:27:34.160240 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:34.160240 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:34.160240 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:34.160240 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:34.160240 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:34.160240 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:34.160240 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:34.160240 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:34.160240 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:34.160240 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:34.160240 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:34.160240 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:27:34.160240 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:27:34.160240 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:27:34.160240 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:27:34.161120 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:27:34.161120 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:27:34.161120 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:27:34.161120 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:27:34.161120 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:34.161120 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:34.161120 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:34.161120 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:34.161120 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:34.161120 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:34.161120 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:34.161120 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:34.161120 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:34.161120 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:34.161120 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:34.161120 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:34.161931 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:34.161931 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:34.161931 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:34.161931 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:34.161931 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:27:34.161931 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:27:34.161931 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:27:34.161931 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:27:34.161931 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:27:34.161931 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:27:34.161931 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:27:34.161931 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:34.161931 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 224/438] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:27:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:27:34.169034 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd4bfe7-bcc9-4faa-8a11-44cdb0d4c1b0 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:34.188999 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:34.190989 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3faf03-3eae-46e6-b10a-8098c4a61126 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:34.234920 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed4a391-a3c1-459f-a098-93b571099ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:34.236602 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4a391-a3c1-459f-a098-93b571099ffa None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:34.236602 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4a391-a3c1-459f-a098-93b571099ffa None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:34.236602 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4a391-a3c1-459f-a098-93b571099ffa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:34.301470 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd4bfe7-bcc9-4faa-8a11-44cdb0d4c1b0 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:34.303522 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:34.320922 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed4a391-a3c1-459f-a098-93b571099ffa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:34.322402 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed4a391-a3c1-459f-a098-93b571099ffa None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:34.345315 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f441cfad-41a4-4512-9957-ce4b948c7662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:34.346187 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441cfad-41a4-4512-9957-ce4b948c7662 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:34.346607 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441cfad-41a4-4512-9957-ce4b948c7662 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:34.346894 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441cfad-41a4-4512-9957-ce4b948c7662 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:34.347651 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f441cfad-41a4-4512-9957-ce4b948c7662 None admin] RBAC: Adding query filter params (name=member) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:34.348076 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f441cfad-41a4-4512-9957-ce4b948c7662 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:34.349578 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f441cfad-41a4-4512-9957-ce4b948c7662 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:34.365374 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 225/439] 10.222.0.13 () {56 vars in 1101 bytes} [Fri Jun 24 08:27:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:27:34.365374 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb052d5d-b391-445d-905f-b6967c910532 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:34.365374 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb052d5d-b391-445d-905f-b6967c910532 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:34.365374 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb052d5d-b391-445d-905f-b6967c910532 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:34.365374 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 226/440] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:34.367173 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb052d5d-b391-445d-905f-b6967c910532 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:34.441858 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 215/441] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:27:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:27:34.443693 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2c50db-6c7b-42c1-8f03-2ae9726d92e1 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:34.444224 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2c50db-6c7b-42c1-8f03-2ae9726d92e1 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:34.444737 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2c50db-6c7b-42c1-8f03-2ae9726d92e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:34.462892 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-bb2c50db-6c7b-42c1-8f03-2ae9726d92e1 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:34.464274 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-bb2c50db-6c7b-42c1-8f03-2ae9726d92e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:34.465095 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:34.500832 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb052d5d-b391-445d-905f-b6967c910532 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:34.531690 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 216/442] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:27:34.532518 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2c50db-6c7b-42c1-8f03-2ae9726d92e1 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:34.541291 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2438db26-8b0a-4a50-b9eb-b9cd479385e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:34.541971 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438db26-8b0a-4a50-b9eb-b9cd479385e2 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:34.542253 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438db26-8b0a-4a50-b9eb-b9cd479385e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:34.542542 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438db26-8b0a-4a50-b9eb-b9cd479385e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:34.553516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:34.610779 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2438db26-8b0a-4a50-b9eb-b9cd479385e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:34.612168 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2438db26-8b0a-4a50-b9eb-b9cd479385e2 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:34.637004 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2c50db-6c7b-42c1-8f03-2ae9726d92e1 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:34.671272 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef0ed44-b3ba-4402-8778-20aecd2e8bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:34.672125 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0ed44-b3ba-4402-8778-20aecd2e8bbf None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:34.672417 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0ed44-b3ba-4402-8778-20aecd2e8bbf None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:34.672690 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0ed44-b3ba-4402-8778-20aecd2e8bbf None admin] PATH_INFO: `/v3/projects/58dfc569119141c688e8c2f5c4faab17` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:34.673961 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef0ed44-b3ba-4402-8778-20aecd2e8bbf None admin] RBAC: Authorizing `identity:get_project(project_id=58dfc569119141c688e8c2f5c4faab17)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:34.675397 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef0ed44-b3ba-4402-8778-20aecd2e8bbf None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:34.677094 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 217/443] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:27:34 2022] GET /identity/v3/projects/58dfc569119141c688e8c2f5c4faab17 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:34.677954 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef0ed44-b3ba-4402-8778-20aecd2e8bbf None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:34.685784 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:34.712604 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 227/444] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:27:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:27:34.713257 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2438db26-8b0a-4a50-b9eb-b9cd479385e2 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:34.763792 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:34.782603 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef0ed44-b3ba-4402-8778-20aecd2e8bbf None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:34.811367 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2438db26-8b0a-4a50-b9eb-b9cd479385e2 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:34.815890 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6bc838-ea25-4068-ab91-3d76c7625b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:34.816673 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6bc838-ea25-4068-ab91-3d76c7625b76 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:34.816953 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6bc838-ea25-4068-ab91-3d76c7625b76 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:34.817224 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6bc838-ea25-4068-ab91-3d76c7625b76 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:34.827872 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6bc838-ea25-4068-ab91-3d76c7625b76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:27:34.835751 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-dc6bc838-ea25-4068-ab91-3d76c7625b76 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:27:34.835751 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:34.835751 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:34.835751 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:34.835751 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:34.835751 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:34.835751 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:34.835751 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:34.835751 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:34.835751 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:34.835751 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:34.835751 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:34.835751 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:27:34.835751 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:27:34.835751 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:27:34.835751 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:27:34.838037 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:27:34.838037 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:27:34.838037 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:27:34.838037 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:27:34.838037 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:34.838037 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:34.838037 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:34.838037 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:34.838037 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:34.838037 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:34.838037 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:34.838037 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:34.838037 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:34.838037 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:34.838037 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:34.838037 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:34.839483 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:34.839483 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:34.839483 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:34.839483 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:34.839483 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:27:34.839483 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:27:34.839483 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:27:34.839483 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:27:34.839483 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:27:34.839483 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:27:34.839483 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:27:34.839483 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:27:34.839483 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 218/445] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Jun 24 08:27:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:34.839483 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6bc838-ea25-4068-ab91-3d76c7625b76 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:34.856291 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60f1e58-877b-4c59-a9dc-ae6a66339877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:34.857147 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f1e58-877b-4c59-a9dc-ae6a66339877 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:34.857429 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f1e58-877b-4c59-a9dc-ae6a66339877 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:34.857777 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f1e58-877b-4c59-a9dc-ae6a66339877 None admin] PATH_INFO: `/v3/groups/5a531630602f4ac0832798f5e5a1855a` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:34.858920 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60f1e58-877b-4c59-a9dc-ae6a66339877 None admin] RBAC: Authorizing `identity:get_group(group_id=5a531630602f4ac0832798f5e5a1855a)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:34.860358 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60f1e58-877b-4c59-a9dc-ae6a66339877 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:34.862114 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 228/446] 10.222.0.13 () {56 vars in 1180 bytes} [Fri Jun 24 08:27:34 2022] GET /identity/v3/groups/5a531630602f4ac0832798f5e5a1855a => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:34.862817 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60f1e58-877b-4c59-a9dc-ae6a66339877 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:34.907793 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:34.941046 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6bc838-ea25-4068-ab91-3d76c7625b76 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:34.964247 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60f1e58-877b-4c59-a9dc-ae6a66339877 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:34.974297 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6f7007-908f-4410-b83c-5f48c6fd4b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:34.975149 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f7007-908f-4410-b83c-5f48c6fd4b14 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:34.975440 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f7007-908f-4410-b83c-5f48c6fd4b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:34.975717 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f7007-908f-4410-b83c-5f48c6fd4b14 None admin] PATH_INFO: `/v3/projects/58dfc569119141c688e8c2f5c4faab17` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:34.976925 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6f7007-908f-4410-b83c-5f48c6fd4b14 None admin] RBAC: Authorizing `identity:get_project(project_id=58dfc569119141c688e8c2f5c4faab17)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:34.978427 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6f7007-908f-4410-b83c-5f48c6fd4b14 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:34.980104 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 219/447] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:27:34 2022] GET /identity/v3/projects/58dfc569119141c688e8c2f5c4faab17 => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:34.980857 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6f7007-908f-4410-b83c-5f48c6fd4b14 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:34.992841 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:35.003255 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d46149-69c2-4020-99b9-bb713c95b48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:35.003924 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d46149-69c2-4020-99b9-bb713c95b48d None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:35.004209 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d46149-69c2-4020-99b9-bb713c95b48d None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:35.004488 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d46149-69c2-4020-99b9-bb713c95b48d None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:35.005236 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d46149-69c2-4020-99b9-bb713c95b48d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:35.005542 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d46149-69c2-4020-99b9-bb713c95b48d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:35.006946 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d46149-69c2-4020-99b9-bb713c95b48d None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:35.015487 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 229/448] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:27:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:35.016254 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d46149-69c2-4020-99b9-bb713c95b48d None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:35.081517 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6f7007-908f-4410-b83c-5f48c6fd4b14 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:35.115130 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9797b5ba-d7ee-4af3-957c-8b4feea13803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:35.115703 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797b5ba-d7ee-4af3-957c-8b4feea13803 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:35.115992 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797b5ba-d7ee-4af3-957c-8b4feea13803 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:35.116249 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d46149-69c2-4020-99b9-bb713c95b48d None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:35.116677 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797b5ba-d7ee-4af3-957c-8b4feea13803 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:35.117237 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9797b5ba-d7ee-4af3-957c-8b4feea13803 None admin] RBAC: Adding query filter params (group.id=5a531630602f4ac0832798f5e5a1855a, role.id=d6ada62a74a94222b864a6c283f8d9b1, scope.project.id=58dfc569119141c688e8c2f5c4faab17) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:35.117628 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9797b5ba-d7ee-4af3-957c-8b4feea13803 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:35.119253 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9797b5ba-d7ee-4af3-957c-8b4feea13803 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:35.119390 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:35.126322 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 220/449] 10.222.0.13 () {56 vars in 1376 bytes} [Fri Jun 24 08:27:34 2022] GET /identity/v3/role_assignments?group.id=5a531630602f4ac0832798f5e5a1855a&scope.project.id=58dfc569119141c688e8c2f5c4faab17&role.id=d6ada62a74a94222b864a6c283f8d9b1 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:35.127127 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9797b5ba-d7ee-4af3-957c-8b4feea13803 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:35.161705 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fe0d49-518f-4a55-8786-bbe37351a19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:35.162727 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe0d49-518f-4a55-8786-bbe37351a19a None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:35.162903 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe0d49-518f-4a55-8786-bbe37351a19a None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:35.163179 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe0d49-518f-4a55-8786-bbe37351a19a None admin] PATH_INFO: `/v3/groups/5a531630602f4ac0832798f5e5a1855a` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:35.164533 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fe0d49-518f-4a55-8786-bbe37351a19a None admin] RBAC: Authorizing `identity:get_group(group_id=5a531630602f4ac0832798f5e5a1855a)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:35.166076 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fe0d49-518f-4a55-8786-bbe37351a19a None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:35.167835 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 230/450] 10.222.0.13 () {56 vars in 1180 bytes} [Fri Jun 24 08:27:35 2022] GET /identity/v3/groups/5a531630602f4ac0832798f5e5a1855a => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:35.173766 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fe0d49-518f-4a55-8786-bbe37351a19a None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:35.223317 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:35.252450 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9797b5ba-d7ee-4af3-957c-8b4feea13803 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:35.291810 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fe0d49-518f-4a55-8786-bbe37351a19a None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:35.294119 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f5632-313a-4ece-87d4-1052d8ef8077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:35.295381 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f5632-313a-4ece-87d4-1052d8ef8077 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:35.295677 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f5632-313a-4ece-87d4-1052d8ef8077 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:35.295955 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f5632-313a-4ece-87d4-1052d8ef8077 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:35.335045 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f8d25d-f851-4205-94d7-1c46ed9489f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:35.335952 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8d25d-f851-4205-94d7-1c46ed9489f7 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:35.336270 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8d25d-f851-4205-94d7-1c46ed9489f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:35.336555 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8d25d-f851-4205-94d7-1c46ed9489f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:35.337391 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f8d25d-f851-4205-94d7-1c46ed9489f7 None admin] RBAC: Adding query filter params (group.id=5a531630602f4ac0832798f5e5a1855a, role.id=bd56838b71ec4a5781fe2df9f3f4f1ab, scope.project.id=58dfc569119141c688e8c2f5c4faab17) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:35.337737 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f8d25d-f851-4205-94d7-1c46ed9489f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:35.339199 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f8d25d-f851-4205-94d7-1c46ed9489f7 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:35.345868 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 231/451] 10.222.0.13 () {56 vars in 1376 bytes} [Fri Jun 24 08:27:35 2022] GET /identity/v3/role_assignments?group.id=5a531630602f4ac0832798f5e5a1855a&scope.project.id=58dfc569119141c688e8c2f5c4faab17&role.id=bd56838b71ec4a5781fe2df9f3f4f1ab => generated 611 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:35.383069 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f5632-313a-4ece-87d4-1052d8ef8077 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:35.385449 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f5632-313a-4ece-87d4-1052d8ef8077 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:35.463655 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:35.490210 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 221/452] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:27:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:27:35.496343 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f8d25d-f851-4205-94d7-1c46ed9489f7 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:35.533855 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:35.601287 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f8d25d-f851-4205-94d7-1c46ed9489f7 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:35.636386 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35112994-7379-4ddd-9b6d-97c75590d19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:35.637180 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35112994-7379-4ddd-9b6d-97c75590d19a None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:35.637463 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35112994-7379-4ddd-9b6d-97c75590d19a None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:35.637804 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35112994-7379-4ddd-9b6d-97c75590d19a None admin] PATH_INFO: `/v3/projects/985b4af5ece540be8554aecad33b1dc4` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:35.639032 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35112994-7379-4ddd-9b6d-97c75590d19a None admin] RBAC: Authorizing `identity:get_project(project_id=985b4af5ece540be8554aecad33b1dc4)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:35.640463 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35112994-7379-4ddd-9b6d-97c75590d19a None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:35.642245 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 232/453] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:27:35 2022] GET /identity/v3/projects/985b4af5ece540be8554aecad33b1dc4 => generated 344 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:35.647750 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f5632-313a-4ece-87d4-1052d8ef8077 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:35.749843 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f5632-313a-4ece-87d4-1052d8ef8077 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:35.785660 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49832c84-bdc8-4ab0-bdb9-dff5c8e41722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:35.786548 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49832c84-bdc8-4ab0-bdb9-dff5c8e41722 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:35.786825 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49832c84-bdc8-4ab0-bdb9-dff5c8e41722 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:35.787098 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49832c84-bdc8-4ab0-bdb9-dff5c8e41722 None admin] PATH_INFO: `/v3/groups/0d3c104b8b6442238675ca64834ba5ed` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:35.788264 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49832c84-bdc8-4ab0-bdb9-dff5c8e41722 None admin] RBAC: Authorizing `identity:get_group(group_id=0d3c104b8b6442238675ca64834ba5ed)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:35.790338 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49832c84-bdc8-4ab0-bdb9-dff5c8e41722 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:35.791955 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 222/454] 10.222.0.13 () {56 vars in 1180 bytes} [Fri Jun 24 08:27:35 2022] GET /identity/v3/groups/0d3c104b8b6442238675ca64834ba5ed => generated 229 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:35.797999 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35112994-7379-4ddd-9b6d-97c75590d19a None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:35.905039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35112994-7379-4ddd-9b6d-97c75590d19a None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:35.907243 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:35.939060 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4421b7ac-5ae6-4bf2-8919-47eecf844b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:35.939703 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4421b7ac-5ae6-4bf2-8919-47eecf844b82 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:35.939991 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4421b7ac-5ae6-4bf2-8919-47eecf844b82 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:35.940272 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4421b7ac-5ae6-4bf2-8919-47eecf844b82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:35.941098 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4421b7ac-5ae6-4bf2-8919-47eecf844b82 None admin] RBAC: Adding query filter params (group.id=0d3c104b8b6442238675ca64834ba5ed, role.id=acfe707eee6944ad94d1915fc9ee7ea3, scope.project.id=985b4af5ece540be8554aecad33b1dc4) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:35.941413 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4421b7ac-5ae6-4bf2-8919-47eecf844b82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:35.942869 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4421b7ac-5ae6-4bf2-8919-47eecf844b82 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:35.949026 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 233/455] 10.222.0.13 () {56 vars in 1376 bytes} [Fri Jun 24 08:27:35 2022] GET /identity/v3/role_assignments?group.id=0d3c104b8b6442238675ca64834ba5ed&scope.project.id=985b4af5ece540be8554aecad33b1dc4&role.id=acfe707eee6944ad94d1915fc9ee7ea3 => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:38.423251 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e83fd6-82df-4fec-b3bf-714e4413cdb6 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:38.424206 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e83fd6-82df-4fec-b3bf-714e4413cdb6 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:38.424696 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e83fd6-82df-4fec-b3bf-714e4413cdb6 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:38.426596 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 223/456] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:38.432197 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a835640-dbf2-4ce1-89d4-bba2c5acb6c8 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:38.432625 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a835640-dbf2-4ce1-89d4-bba2c5acb6c8 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:38.432822 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a835640-dbf2-4ce1-89d4-bba2c5acb6c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:38.457874 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-5a835640-dbf2-4ce1-89d4-bba2c5acb6c8 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:38.459090 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-5a835640-dbf2-4ce1-89d4-bba2c5acb6c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:38.551255 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e1bc6-0e99-47cf-ba4a-397de1e44092 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:38.551887 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e1bc6-0e99-47cf-ba4a-397de1e44092 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:38.552560 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e1bc6-0e99-47cf-ba4a-397de1e44092 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:38.554622 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 224/457] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:38.560852 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7105a-52aa-4621-9395-d7362454b8a9 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:38.561719 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7105a-52aa-4621-9395-d7362454b8a9 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:38.562488 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7105a-52aa-4621-9395-d7362454b8a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:38.573589 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 234/458] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:27:38.594735 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-22d7105a-52aa-4621-9395-d7362454b8a9 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:38.595930 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-22d7105a-52aa-4621-9395-d7362454b8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:38.602688 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a835640-dbf2-4ce1-89d4-bba2c5acb6c8 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:38.674042 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:38.737465 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 225/459] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 08:27:38.750922 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a835640-dbf2-4ce1-89d4-bba2c5acb6c8 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:38.754088 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:38.806925 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7105a-52aa-4621-9395-d7362454b8a9 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:38.809509 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab0ce1b-1bf4-4212-b025-2559499ed659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:38.810568 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab0ce1b-1bf4-4212-b025-2559499ed659 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:38.811080 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab0ce1b-1bf4-4212-b025-2559499ed659 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:38.811554 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab0ce1b-1bf4-4212-b025-2559499ed659 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:38.823692 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab0ce1b-1bf4-4212-b025-2559499ed659 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:38.825487 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab0ce1b-1bf4-4212-b025-2559499ed659 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:38.844775 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-eab0ce1b-1bf4-4212-b025-2559499ed659 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:38.844775 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:38.844775 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:38.844775 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:38.844775 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:38.844775 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:38.844775 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:38.844775 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:38.844775 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:38.844775 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:38.844775 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:38.844775 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:38.844775 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:27:38.844775 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:27:38.844775 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:27:38.845713 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:27:38.845713 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:38.845713 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:38.845713 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:38.845713 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:38.845713 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:38.845713 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:38.845713 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:38.845713 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:38.845713 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:38.845713 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:38.845713 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:38.845713 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:38.845713 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:38.845713 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:38.845713 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:38.845713 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:38.846584 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:27:38.846584 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:27:38.846584 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:38.846584 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:38.848874 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 235/460] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:27:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:27:38.854484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab0ce1b-1bf4-4212-b025-2559499ed659 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:38.967048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab0ce1b-1bf4-4212-b025-2559499ed659 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:39.004223 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b30cfb-9e33-48c1-9e16-4c696c420948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:39.005323 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b30cfb-9e33-48c1-9e16-4c696c420948 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:39.005848 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b30cfb-9e33-48c1-9e16-4c696c420948 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:39.006323 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b30cfb-9e33-48c1-9e16-4c696c420948 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:39.007269 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b30cfb-9e33-48c1-9e16-4c696c420948 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:39.007788 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b30cfb-9e33-48c1-9e16-4c696c420948 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:39.009982 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b30cfb-9e33-48c1-9e16-4c696c420948 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:39.024614 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 236/461] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:27:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:27:39.029616 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b30cfb-9e33-48c1-9e16-4c696c420948 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:39.030450 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7105a-52aa-4621-9395-d7362454b8a9 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:39.050047 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:39.110690 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c963fa-ee63-48ed-98da-4c1358e97c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:39.110910 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c963fa-ee63-48ed-98da-4c1358e97c73 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:39.111041 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c963fa-ee63-48ed-98da-4c1358e97c73 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:39.111347 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c963fa-ee63-48ed-98da-4c1358e97c73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:39.122674 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c963fa-ee63-48ed-98da-4c1358e97c73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:39.124150 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c963fa-ee63-48ed-98da-4c1358e97c73 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:39.142845 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-55c963fa-ee63-48ed-98da-4c1358e97c73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:39.142845 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:39.142845 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:39.142845 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:39.142845 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:39.142845 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:39.142845 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:39.142845 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:39.142845 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:39.142845 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:39.142845 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:39.142845 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:39.142845 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:27:39.142845 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:27:39.142845 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:27:39.143738 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:27:39.143738 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:39.143738 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:39.143738 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:39.143738 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:39.143738 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:39.143738 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:39.143738 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:39.143738 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:39.143738 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:39.143738 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:39.143738 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:39.143738 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:39.143738 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:39.143738 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:39.143738 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:39.143738 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:39.144587 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:27:39.144587 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:27:39.144587 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:39.144587 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:27:39.144587 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 226/462] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:27:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:27:39.150363 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7965fe-4fc2-456e-be64-7d3c314ee10a None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:39.150363 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7965fe-4fc2-456e-be64-7d3c314ee10a None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:39.150363 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7965fe-4fc2-456e-be64-7d3c314ee10a None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:39.150363 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 227/463] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:39.151334 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b30cfb-9e33-48c1-9e16-4c696c420948 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:39.152365 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7965fe-4fc2-456e-be64-7d3c314ee10a None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:39.178668 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:39.190301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cf5499-54b0-4a47-a0f2-71cc3ed253d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:39.191165 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf5499-54b0-4a47-a0f2-71cc3ed253d8 None admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:39.191457 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf5499-54b0-4a47-a0f2-71cc3ed253d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:39.191738 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf5499-54b0-4a47-a0f2-71cc3ed253d8 None admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:39.193230 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cf5499-54b0-4a47-a0f2-71cc3ed253d8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:39.194719 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cf5499-54b0-4a47-a0f2-71cc3ed253d8 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:39.225795 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 237/464] 10.222.0.13 () {60 vars in 1132 bytes} [Fri Jun 24 08:27:39 2022] POST /identity/v3/users => generated 246 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:27:39.225795 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e74ce9-6a4e-4340-ae6b-747785d407e6 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:39.225795 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e74ce9-6a4e-4340-ae6b-747785d407e6 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:39.225795 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e74ce9-6a4e-4340-ae6b-747785d407e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:39.253002 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-b6e74ce9-6a4e-4340-ae6b-747785d407e6 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:39.254634 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b6e74ce9-6a4e-4340-ae6b-747785d407e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:39.270575 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7965fe-4fc2-456e-be64-7d3c314ee10a None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:39.320375 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d530bb1e-f3be-44f5-8a71-fd636f8109fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:39.321619 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d530bb1e-f3be-44f5-8a71-fd636f8109fa None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:39.321939 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d530bb1e-f3be-44f5-8a71-fd636f8109fa None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:39.322224 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d530bb1e-f3be-44f5-8a71-fd636f8109fa None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:39.322966 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d530bb1e-f3be-44f5-8a71-fd636f8109fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:39.323277 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d530bb1e-f3be-44f5-8a71-fd636f8109fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:39.324864 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d530bb1e-f3be-44f5-8a71-fd636f8109fa None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:39.342574 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:39.348350 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 228/465] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:27:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:39.349769 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72ce4c-a0cf-4d06-8dc3-291d6bcfcaac None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:39.353743 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72ce4c-a0cf-4d06-8dc3-291d6bcfcaac None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:39.353743 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72ce4c-a0cf-4d06-8dc3-291d6bcfcaac None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:39.353743 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 229/466] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:39.355200 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 238/467] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:27:39.369142 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d72ce4c-a0cf-4d06-8dc3-291d6bcfcaac None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:39.374202 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca95f0-71ce-4614-8802-b11c40b8a0cf None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:39.374202 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca95f0-71ce-4614-8802-b11c40b8a0cf None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:39.374202 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca95f0-71ce-4614-8802-b11c40b8a0cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:39.411878 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-6cca95f0-71ce-4614-8802-b11c40b8a0cf None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:39.413114 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-6cca95f0-71ce-4614-8802-b11c40b8a0cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:39.515604 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:39.518062 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 239/468] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:27:39.519012 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cca95f0-71ce-4614-8802-b11c40b8a0cf None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:39.540007 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d72ce4c-a0cf-4d06-8dc3-291d6bcfcaac None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:39.595389 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c7747e-4f09-48ff-a415-51229c90f0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:39.596737 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c7747e-4f09-48ff-a415-51229c90f0ee None admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:39.597226 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c7747e-4f09-48ff-a415-51229c90f0ee None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:39.597854 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c7747e-4f09-48ff-a415-51229c90f0ee None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:39.598608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c7747e-4f09-48ff-a415-51229c90f0ee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:39.600115 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c7747e-4f09-48ff-a415-51229c90f0ee None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:39.629422 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 230/469] 10.222.0.13 () {60 vars in 1132 bytes} [Fri Jun 24 08:27:39 2022] POST /identity/v3/users => generated 248 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:27:39.630027 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c7747e-4f09-48ff-a415-51229c90f0ee None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:39.676386 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cca95f0-71ce-4614-8802-b11c40b8a0cf None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:39.714060 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:39.725285 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c248dcc-60ba-4b9e-95c3-a40ce71e2c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:39.726618 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c248dcc-60ba-4b9e-95c3-a40ce71e2c1c None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:39.727488 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c248dcc-60ba-4b9e-95c3-a40ce71e2c1c None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:39.728549 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c248dcc-60ba-4b9e-95c3-a40ce71e2c1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:39.744699 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c248dcc-60ba-4b9e-95c3-a40ce71e2c1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:39.746650 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c248dcc-60ba-4b9e-95c3-a40ce71e2c1c None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:39.771928 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-6c248dcc-60ba-4b9e-95c3-a40ce71e2c1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:39.771928 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:39.771928 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:39.771928 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:39.771928 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:39.771928 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:39.771928 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:39.771928 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:39.771928 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:39.771928 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:39.771928 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:39.771928 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:39.771928 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:27:39.771928 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:27:39.771928 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:27:39.772821 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:27:39.772821 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:39.772821 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:39.772821 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:39.772821 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:39.772821 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:39.772821 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:39.772821 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:39.772821 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:39.772821 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:39.772821 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:39.772821 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:39.772821 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:39.772821 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:39.772821 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:39.772821 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:39.772821 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:39.773672 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:27:39.773672 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:27:39.773672 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:39.773672 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:39.785937 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 240/470] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:27:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:39.788127 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c248dcc-60ba-4b9e-95c3-a40ce71e2c1c None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:39.861892 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c7747e-4f09-48ff-a415-51229c90f0ee None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:39.912156 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe61bab-10b6-4cae-a911-ab1924a55891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:39.913268 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe61bab-10b6-4cae-a911-ab1924a55891 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:39.913560 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe61bab-10b6-4cae-a911-ab1924a55891 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:39.913871 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe61bab-10b6-4cae-a911-ab1924a55891 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:39.917307 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c248dcc-60ba-4b9e-95c3-a40ce71e2c1c None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:39.927776 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:39.930584 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe61bab-10b6-4cae-a911-ab1924a55891 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:39.937280 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe61bab-10b6-4cae-a911-ab1924a55891 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:39.960455 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-4fe61bab-10b6-4cae-a911-ab1924a55891 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:39.960455 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:39.960455 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:39.960455 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:39.960455 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:39.960455 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:39.960455 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:39.960455 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:39.960455 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:39.960455 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:39.960455 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:39.960455 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:39.960455 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:27:39.960455 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:27:39.960455 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:27:39.961829 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:27:39.961829 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:39.961829 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:39.961829 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:39.961829 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:39.961829 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:39.961829 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:39.961829 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:39.961829 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:39.961829 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:39.961829 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:39.961829 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:39.961829 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:39.961829 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:39.961829 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:39.961829 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:39.961829 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:39.963617 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:27:39.963617 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:27:39.963617 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:39.963617 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:27:39.963617 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 231/471] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:27:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:39.963617 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f2a91a-aec1-46db-9a07-a61d485a1c32 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:39.963617 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f2a91a-aec1-46db-9a07-a61d485a1c32 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:39.964075 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f2a91a-aec1-46db-9a07-a61d485a1c32 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:39.965245 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 232/472] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:39.970809 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30bcca2-51d2-43c1-8096-31fb37b8e6c9 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:39.971136 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30bcca2-51d2-43c1-8096-31fb37b8e6c9 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:39.971354 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30bcca2-51d2-43c1-8096-31fb37b8e6c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:39.971641 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21bdaa9-774c-4345-8ba5-93364bd61139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:39.972306 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21bdaa9-774c-4345-8ba5-93364bd61139 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:39.972666 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21bdaa9-774c-4345-8ba5-93364bd61139 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:39.972866 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21bdaa9-774c-4345-8ba5-93364bd61139 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:39.973625 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21bdaa9-774c-4345-8ba5-93364bd61139 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:39.973939 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21bdaa9-774c-4345-8ba5-93364bd61139 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:39.975326 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21bdaa9-774c-4345-8ba5-93364bd61139 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:39.989002 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 241/473] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:27:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:39.990174 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21bdaa9-774c-4345-8ba5-93364bd61139 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:39.994256 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-c30bcca2-51d2-43c1-8096-31fb37b8e6c9 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:39.995628 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-c30bcca2-51d2-43c1-8096-31fb37b8e6c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:40.042096 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:40.070314 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 233/474] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 08:27:40.071653 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30bcca2-51d2-43c1-8096-31fb37b8e6c9 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:40.131798 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21bdaa9-774c-4345-8ba5-93364bd61139 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:40.178312 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab7b1b9-7ba9-4029-925f-258ad0a35281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:40.179182 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab7b1b9-7ba9-4029-925f-258ad0a35281 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:40.179564 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab7b1b9-7ba9-4029-925f-258ad0a35281 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:40.179935 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab7b1b9-7ba9-4029-925f-258ad0a35281 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:40.181619 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab7b1b9-7ba9-4029-925f-258ad0a35281 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:40.182047 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab7b1b9-7ba9-4029-925f-258ad0a35281 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:40.183719 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab7b1b9-7ba9-4029-925f-258ad0a35281 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:40.201351 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 242/475] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:27:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:40.207011 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab7b1b9-7ba9-4029-925f-258ad0a35281 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:40.231539 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30bcca2-51d2-43c1-8096-31fb37b8e6c9 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:40.269169 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:40.289980 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2992b6-691b-4e72-8083-b3726c3ea830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:40.291921 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2992b6-691b-4e72-8083-b3726c3ea830 None admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:40.292510 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2992b6-691b-4e72-8083-b3726c3ea830 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:40.293252 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2992b6-691b-4e72-8083-b3726c3ea830 None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:40.294537 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2992b6-691b-4e72-8083-b3726c3ea830 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:40.297155 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2992b6-691b-4e72-8083-b3726c3ea830 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:40.338434 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 234/476] 10.222.0.13 () {60 vars in 1132 bytes} [Fri Jun 24 08:27:40 2022] POST /identity/v3/users => generated 248 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:27:40.339018 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2992b6-691b-4e72-8083-b3726c3ea830 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:40.362143 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:40.368403 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab7b1b9-7ba9-4029-925f-258ad0a35281 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:40.455763 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e843f2-00c1-4b78-bed5-70052b22cae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:40.456451 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e843f2-00c1-4b78-bed5-70052b22cae3 None admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:40.456731 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e843f2-00c1-4b78-bed5-70052b22cae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:40.457010 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e843f2-00c1-4b78-bed5-70052b22cae3 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:40.457794 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e843f2-00c1-4b78-bed5-70052b22cae3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:40.460900 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e843f2-00c1-4b78-bed5-70052b22cae3 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:40.477235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.sql.core [None req-a7e843f2-00c1-4b78-bed5-70052b22cae3 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 24 08:27:40.477235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 24 08:27:40.477235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [parameters: {'id': 'fad8377f746941949752233a806f79ce', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 24 08:27:40.477235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80424) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 24 08:27:40.509878 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-a7e843f2-00c1-4b78-bed5-70052b22cae3 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 24 08:27:40.509878 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:40.509878 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 08:27:40.509878 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 08:27:40.509878 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 08:27:40.509878 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 08:27:40.509878 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 08:27:40.509878 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 08:27:40.509878 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 08:27:40.509878 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application conn.query(q) Jun 24 08:27:40.509878 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 08:27:40.509878 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 08:27:40.509878 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 08:27:40.509878 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application result.read() Jun 24 08:27:40.509878 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 08:27:40.510888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 08:27:40.510888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 08:27:40.510888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 08:27:40.510888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 08:27:40.510888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 08:27:40.510888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 08:27:40.510888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 08:27:40.510888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 24 08:27:40.510888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:40.510888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 24 08:27:40.510888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:40.510888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:40.510888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 24 08:27:40.510888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 24 08:27:40.510888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 24 08:27:40.510888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return ref.to_dict() Jun 24 08:27:40.510888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 08:27:40.510888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application next(self.gen) Jun 24 08:27:40.511792 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 24 08:27:40.511792 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application yield resource Jun 24 08:27:40.511792 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 08:27:40.511792 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application next(self.gen) Jun 24 08:27:40.511792 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 24 08:27:40.511792 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application self.session.rollback() Jun 24 08:27:40.511792 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 24 08:27:40.511792 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application self.force_reraise() Jun 24 08:27:40.511792 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 24 08:27:40.511792 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise self.value Jun 24 08:27:40.511792 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 24 08:27:40.511792 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 24 08:27:40.511792 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 24 08:27:40.511792 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application session.commit() Jun 24 08:27:40.511792 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 24 08:27:40.511792 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 24 08:27:40.511792 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 24 08:27:40.513419 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application self._prepare_impl() Jun 24 08:27:40.513419 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 24 08:27:40.513419 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application self.session.flush() Jun 24 08:27:40.513419 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 24 08:27:40.513419 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application self._flush(objects) Jun 24 08:27:40.513419 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 24 08:27:40.513419 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 24 08:27:40.513419 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 24 08:27:40.513419 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application compat.raise_( Jun 24 08:27:40.513419 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 08:27:40.513419 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception Jun 24 08:27:40.513419 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 24 08:27:40.513419 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application flush_context.execute() Jun 24 08:27:40.513419 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 24 08:27:40.513419 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rec.execute(self) Jun 24 08:27:40.513419 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 24 08:27:40.513419 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 24 08:27:40.514381 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 24 08:27:40.514381 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 24 08:27:40.514381 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 24 08:27:40.514381 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 24 08:27:40.514381 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 24 08:27:40.514381 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 24 08:27:40.514381 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 24 08:27:40.514381 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 24 08:27:40.514381 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 24 08:27:40.514381 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 24 08:27:40.514381 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 24 08:27:40.514381 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 24 08:27:40.514381 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 24 08:27:40.514381 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 24 08:27:40.514381 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 08:27:40.514381 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception Jun 24 08:27:40.516018 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 08:27:40.516018 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 08:27:40.516018 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 08:27:40.516018 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 08:27:40.516018 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 08:27:40.516018 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 08:27:40.516018 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 08:27:40.516018 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application conn.query(q) Jun 24 08:27:40.516018 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 08:27:40.516018 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 08:27:40.516018 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 08:27:40.516018 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application result.read() Jun 24 08:27:40.516018 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 08:27:40.516018 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 08:27:40.516018 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 08:27:40.516018 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 08:27:40.518039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 08:27:40.518039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 08:27:40.518039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 08:27:40.518039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 08:27:40.518039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 24 08:27:40.518039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 24 08:27:40.518039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [parameters: {'id': 'fad8377f746941949752233a806f79ce', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 24 08:27:40.518039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 24 08:27:40.518039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:40.518039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:27:40.518039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:40.518039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:40.518039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:40.518039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:40.518039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:40.518039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:40.520418 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:40.520418 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:40.520418 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:40.520418 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:40.520418 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:40.520418 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:40.520418 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 24 08:27:40.520418 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 24 08:27:40.520418 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:40.520418 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:40.520418 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 24 08:27:40.520418 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 24 08:27:40.520418 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 24 08:27:40.520418 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 24 08:27:40.520418 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 24 08:27:40.520418 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:40.522376 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 243/477] 10.222.0.13 () {60 vars in 1131 bytes} [Fri Jun 24 08:27:40 2022] POST /identity/v3/roles => generated 144 bytes in 310 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:40.522507 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e843f2-00c1-4b78-bed5-70052b22cae3 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:40.564421 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2992b6-691b-4e72-8083-b3726c3ea830 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:40.605387 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e9624-ce8d-4373-ba3f-c641ee397f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:40.606359 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e9624-ce8d-4373-ba3f-c641ee397f79 None admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:40.606813 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e9624-ce8d-4373-ba3f-c641ee397f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:40.607282 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e9624-ce8d-4373-ba3f-c641ee397f79 None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:40.608156 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6e9624-ce8d-4373-ba3f-c641ee397f79 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:40.609668 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6e9624-ce8d-4373-ba3f-c641ee397f79 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:40.633737 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 235/478] 10.222.0.13 () {60 vars in 1132 bytes} [Fri Jun 24 08:27:40 2022] POST /identity/v3/users => generated 249 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:27:40.658127 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e843f2-00c1-4b78-bed5-70052b22cae3 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:40.700738 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a945985-4f49-4698-ac52-1038aaeb6eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:40.701851 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a945985-4f49-4698-ac52-1038aaeb6eb9 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:40.702274 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a945985-4f49-4698-ac52-1038aaeb6eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:40.702746 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a945985-4f49-4698-ac52-1038aaeb6eb9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:40.716186 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a945985-4f49-4698-ac52-1038aaeb6eb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:27:40.725077 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-3a945985-4f49-4698-ac52-1038aaeb6eb9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 08:27:40.725077 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:40.725077 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:40.725077 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:40.725077 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:40.725077 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:40.725077 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:40.725077 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:40.725077 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:40.725077 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:40.725077 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:40.725077 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:40.725077 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:27:40.725077 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:27:40.725077 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:27:40.725077 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:27:40.726242 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:27:40.726242 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:27:40.726242 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:27:40.726242 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:27:40.726242 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:40.726242 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:40.726242 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:40.726242 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:40.726242 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:40.726242 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:40.726242 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:40.726242 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:40.726242 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:40.726242 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:40.726242 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:40.726242 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:40.727278 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:40.727278 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:40.727278 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:40.727278 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:40.727278 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:27:40.727278 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:27:40.727278 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:27:40.727278 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:27:40.727278 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:27:40.727278 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:27:40.727278 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 08:27:40.727278 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:40.729871 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 244/479] 10.222.0.13 () {56 vars in 1114 bytes} [Fri Jun 24 08:27:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:40.735755 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e9624-ce8d-4373-ba3f-c641ee397f79 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:40.782758 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:40.853909 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e9624-ce8d-4373-ba3f-c641ee397f79 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:40.889770 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed960da3-9b1a-440d-9d96-5a0a4f801617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:40.890444 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed960da3-9b1a-440d-9d96-5a0a4f801617 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:40.890730 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed960da3-9b1a-440d-9d96-5a0a4f801617 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:40.891011 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed960da3-9b1a-440d-9d96-5a0a4f801617 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:40.891846 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed960da3-9b1a-440d-9d96-5a0a4f801617 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:40.892156 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed960da3-9b1a-440d-9d96-5a0a4f801617 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:40.893555 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed960da3-9b1a-440d-9d96-5a0a4f801617 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:40.902521 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 236/480] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:27:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:41.709377 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575336e4-c64e-47c6-89a9-e5a283aa95f5 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:41.710265 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575336e4-c64e-47c6-89a9-e5a283aa95f5 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:41.710682 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575336e4-c64e-47c6-89a9-e5a283aa95f5 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:41.712804 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 245/481] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:41.718491 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a183ec3-5b91-4087-a50b-550889afe772 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:41.718930 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a183ec3-5b91-4087-a50b-550889afe772 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:41.719121 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a183ec3-5b91-4087-a50b-550889afe772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:41.746729 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-6a183ec3-5b91-4087-a50b-550889afe772 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:41.747942 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-6a183ec3-5b91-4087-a50b-550889afe772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:41.843963 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 237/482] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:27:41.869585 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575336e4-c64e-47c6-89a9-e5a283aa95f5 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:41.917278 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:42.023395 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575336e4-c64e-47c6-89a9-e5a283aa95f5 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:42.067524 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facb03ca-99a5-41e4-bae3-682ca102187c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:42.068367 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb03ca-99a5-41e4-bae3-682ca102187c None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:42.068655 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb03ca-99a5-41e4-bae3-682ca102187c None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:42.068932 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb03ca-99a5-41e4-bae3-682ca102187c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:42.087132 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facb03ca-99a5-41e4-bae3-682ca102187c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:27:42.095229 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-facb03ca-99a5-41e4-bae3-682ca102187c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:27:42.095229 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:42.095229 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:42.095229 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:42.095229 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:42.095229 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:42.095229 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:42.095229 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:42.095229 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:42.095229 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:42.095229 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:42.095229 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:42.095229 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:27:42.095229 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:27:42.095229 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:27:42.095229 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:27:42.096172 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:27:42.096172 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:27:42.096172 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:27:42.096172 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:27:42.096172 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:42.096172 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:42.096172 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:42.096172 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:42.096172 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:42.096172 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:42.096172 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:42.096172 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:42.096172 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:42.096172 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:42.096172 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:42.096172 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:42.097542 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:42.097542 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:42.097542 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:42.097542 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:42.097542 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:27:42.097542 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:27:42.097542 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:27:42.097542 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:27:42.097542 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:27:42.097542 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:27:42.097542 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:27:42.097542 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:42.097542 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 246/483] 10.222.0.13 () {56 vars in 1102 bytes} [Fri Jun 24 08:27:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:42.102518 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a183ec3-5b91-4087-a50b-550889afe772 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:42.148708 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:42.245427 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a183ec3-5b91-4087-a50b-550889afe772 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:42.250593 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75395f6e-335d-40a6-8b81-9c9dc8161c7b None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:42.251131 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75395f6e-335d-40a6-8b81-9c9dc8161c7b None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:42.251646 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75395f6e-335d-40a6-8b81-9c9dc8161c7b None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:42.253497 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 247/484] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:42.272673 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07890993-6c97-497e-ace9-04f90b230982 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:42.273255 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07890993-6c97-497e-ace9-04f90b230982 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:42.274325 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07890993-6c97-497e-ace9-04f90b230982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:42.285674 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fade0a-18bd-44fa-b9b6-80817112d452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:42.286366 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fade0a-18bd-44fa-b9b6-80817112d452 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:42.286648 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fade0a-18bd-44fa-b9b6-80817112d452 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:42.286924 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fade0a-18bd-44fa-b9b6-80817112d452 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:42.287660 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fade0a-18bd-44fa-b9b6-80817112d452 None admin] RBAC: Adding query filter params (name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:42.288058 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fade0a-18bd-44fa-b9b6-80817112d452 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:42.289730 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fade0a-18bd-44fa-b9b6-80817112d452 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:42.299793 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 238/485] 10.222.0.13 () {56 vars in 1103 bytes} [Fri Jun 24 08:27:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:42.309378 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fade0a-18bd-44fa-b9b6-80817112d452 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:42.315703 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-07890993-6c97-497e-ace9-04f90b230982 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:42.317144 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-07890993-6c97-497e-ace9-04f90b230982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:42.414194 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:42.429065 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fade0a-18bd-44fa-b9b6-80817112d452 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:42.463823 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 248/486] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:27:42.467216 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0dc946-f117-4b76-807f-ff778961161c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:42.468015 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0dc946-f117-4b76-807f-ff778961161c None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:42.468305 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0dc946-f117-4b76-807f-ff778961161c None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:42.468586 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0dc946-f117-4b76-807f-ff778961161c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:42.485932 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0dc946-f117-4b76-807f-ff778961161c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:42.487393 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0dc946-f117-4b76-807f-ff778961161c None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:42.488843 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07890993-6c97-497e-ace9-04f90b230982 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:42.512670 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:42.528185 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-1f0dc946-f117-4b76-807f-ff778961161c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:42.528185 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:42.528185 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:42.528185 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:42.528185 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:42.528185 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:42.528185 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:42.528185 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:42.528185 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:42.528185 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:42.528185 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:42.528185 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:42.528185 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:27:42.528185 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:27:42.528185 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:27:42.529035 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:27:42.529035 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:42.529035 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:42.529035 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:42.529035 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:42.529035 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:42.529035 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:42.529035 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:42.529035 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:42.529035 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:42.529035 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:42.529035 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:42.529035 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:42.529035 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:42.529035 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:42.529035 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:42.529035 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:42.529899 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:27:42.529899 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:27:42.529899 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:42.529899 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:27:42.537091 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 239/487] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:27:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:27:42.543067 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0dc946-f117-4b76-807f-ff778961161c None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:42.679968 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07890993-6c97-497e-ace9-04f90b230982 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:42.697033 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0dc946-f117-4b76-807f-ff778961161c None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:42.737076 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab41ab50-cc5b-48fc-9381-cd25302d228c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:42.737893 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41ab50-cc5b-48fc-9381-cd25302d228c None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:42.738068 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41ab50-cc5b-48fc-9381-cd25302d228c None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:42.738310 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41ab50-cc5b-48fc-9381-cd25302d228c None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:42.739039 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab41ab50-cc5b-48fc-9381-cd25302d228c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:42.739443 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab41ab50-cc5b-48fc-9381-cd25302d228c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:42.740843 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab41ab50-cc5b-48fc-9381-cd25302d228c None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:42.754700 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 240/488] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:27:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:42.756446 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:42.759174 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47adbb-9f5c-41a1-88a5-a3649a88883f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:42.760734 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47adbb-9f5c-41a1-88a5-a3649a88883f None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:42.761853 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab41ab50-cc5b-48fc-9381-cd25302d228c None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:42.763153 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47adbb-9f5c-41a1-88a5-a3649a88883f None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:42.765561 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47adbb-9f5c-41a1-88a5-a3649a88883f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:42.786302 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be47adbb-9f5c-41a1-88a5-a3649a88883f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:27:42.803106 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-be47adbb-9f5c-41a1-88a5-a3649a88883f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:27:42.803106 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:42.803106 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:42.803106 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:42.803106 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:42.803106 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:42.803106 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:42.803106 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:42.803106 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:42.803106 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:42.803106 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:42.803106 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:42.803106 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:27:42.803106 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:27:42.803106 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:27:42.803106 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:27:42.804063 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:27:42.804063 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:27:42.804063 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:27:42.804063 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:27:42.804063 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:42.804063 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:42.804063 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:42.804063 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:42.804063 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:42.804063 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:42.804063 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:42.804063 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:42.804063 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:42.804063 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:42.804063 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:42.804063 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:42.806062 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:42.806062 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:42.806062 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:42.806062 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:42.806062 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:27:42.806062 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:27:42.806062 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:27:42.806062 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:27:42.806062 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:27:42.806062 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:27:42.806062 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:27:42.806062 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:42.806062 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 249/489] 10.222.0.13 () {56 vars in 1102 bytes} [Fri Jun 24 08:27:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:27:42.811239 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47adbb-9f5c-41a1-88a5-a3649a88883f None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:42.884864 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab41ab50-cc5b-48fc-9381-cd25302d228c None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:42.949948 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ecf64-e6ea-4951-b54d-110b221db91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:42.951168 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ecf64-e6ea-4951-b54d-110b221db91b None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:42.951670 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ecf64-e6ea-4951-b54d-110b221db91b None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:42.952156 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ecf64-e6ea-4951-b54d-110b221db91b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:42.960695 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902ecf64-e6ea-4951-b54d-110b221db91b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:42.963685 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902ecf64-e6ea-4951-b54d-110b221db91b None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:42.981409 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:42.982737 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-902ecf64-e6ea-4951-b54d-110b221db91b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 08:27:42.982737 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:42.982737 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:42.982737 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:42.982737 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:42.982737 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:42.982737 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:42.982737 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:42.982737 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:42.982737 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:42.982737 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:42.982737 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:42.982737 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:27:42.982737 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:27:42.982737 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:27:42.982737 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:27:42.983608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:42.983608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:42.983608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:27:42.983608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:27:42.983608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:27:42.983608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:27:42.983608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:42.983608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:42.983608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:42.983608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:42.983608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:42.983608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:42.983608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:42.983608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:42.983608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:42.983608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:42.983608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:42.983608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:42.984492 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:42.984492 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:42.984492 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:27:42.984492 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:27:42.984492 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:27:42.984492 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:27:42.984492 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:27:42.984492 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:27:42.984492 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 08:27:42.984492 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:27:42.987172 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 241/490] 10.222.0.13 () {56 vars in 1093 bytes} [Fri Jun 24 08:27:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:42.995098 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ecf64-e6ea-4951-b54d-110b221db91b None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:43.044011 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47adbb-9f5c-41a1-88a5-a3649a88883f None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:43.101123 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:43.122473 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bd1212-d1ff-4e97-8540-e903c7e79f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:43.123163 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bd1212-d1ff-4e97-8540-e903c7e79f28 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:43.123459 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bd1212-d1ff-4e97-8540-e903c7e79f28 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:43.123735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bd1212-d1ff-4e97-8540-e903c7e79f28 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:43.124504 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bd1212-d1ff-4e97-8540-e903c7e79f28 None admin] RBAC: Adding query filter params (name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:43.124813 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bd1212-d1ff-4e97-8540-e903c7e79f28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:43.126542 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bd1212-d1ff-4e97-8540-e903c7e79f28 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:43.137327 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 250/491] 10.222.0.13 () {56 vars in 1103 bytes} [Fri Jun 24 08:27:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:27:43.143063 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bd1212-d1ff-4e97-8540-e903c7e79f28 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:43.179442 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ecf64-e6ea-4951-b54d-110b221db91b None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:43.224989 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eb2552-c0d5-435e-91db-99d93c598988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:43.225857 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb2552-c0d5-435e-91db-99d93c598988 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:43.226351 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb2552-c0d5-435e-91db-99d93c598988 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:43.227369 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb2552-c0d5-435e-91db-99d93c598988 None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:43.228782 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eb2552-c0d5-435e-91db-99d93c598988 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:43.229105 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eb2552-c0d5-435e-91db-99d93c598988 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:43.231235 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eb2552-c0d5-435e-91db-99d93c598988 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:43.276825 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 242/492] 10.222.0.13 () {56 vars in 1133 bytes} [Fri Jun 24 08:27:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:43.278283 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691610a3-1af2-4c9d-bed9-0dc3f0083e2a None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:43.278574 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691610a3-1af2-4c9d-bed9-0dc3f0083e2a None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:43.278848 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691610a3-1af2-4c9d-bed9-0dc3f0083e2a None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:43.284171 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 243/493] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:43.284171 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691610a3-1af2-4c9d-bed9-0dc3f0083e2a None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:43.302346 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bd1212-d1ff-4e97-8540-e903c7e79f28 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:43.345359 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031885c7-9fc2-482a-9a38-e9dcb0b9b8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:43.346185 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031885c7-9fc2-482a-9a38-e9dcb0b9b8e0 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:43.346469 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031885c7-9fc2-482a-9a38-e9dcb0b9b8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:43.346997 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031885c7-9fc2-482a-9a38-e9dcb0b9b8e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:43.349843 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:43.362656 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031885c7-9fc2-482a-9a38-e9dcb0b9b8e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:43.364117 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031885c7-9fc2-482a-9a38-e9dcb0b9b8e0 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:43.402908 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691610a3-1af2-4c9d-bed9-0dc3f0083e2a None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:43.410671 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-031885c7-9fc2-482a-9a38-e9dcb0b9b8e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:43.410671 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:43.410671 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:43.410671 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:43.410671 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:43.410671 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:43.410671 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:43.410671 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:43.410671 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:43.410671 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:43.410671 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:43.410671 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:43.410671 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:27:43.410671 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:27:43.410671 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:27:43.411537 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:27:43.411537 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:43.411537 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:43.411537 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:43.411537 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:43.411537 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:43.411537 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:43.411537 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:43.411537 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:43.411537 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:43.411537 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:43.411537 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:43.411537 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:43.411537 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:43.411537 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:43.411537 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:43.411537 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:43.412359 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:27:43.412359 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:27:43.412359 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:43.412359 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:43.413042 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 251/494] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:27:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:43.417183 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c96be8c-7938-4a96-9f5a-0995c2328891 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:43.417183 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c96be8c-7938-4a96-9f5a-0995c2328891 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:43.417183 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c96be8c-7938-4a96-9f5a-0995c2328891 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:43.419141 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 252/495] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:43.421178 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca237b2-ac2e-4bfb-98eb-f440a9df69d1 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:43.421681 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca237b2-ac2e-4bfb-98eb-f440a9df69d1 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:43.422213 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca237b2-ac2e-4bfb-98eb-f440a9df69d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:43.423738 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:43.442335 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5952d3-81d2-44fe-b91b-1a4bae7cbfa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:43.443426 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5952d3-81d2-44fe-b91b-1a4bae7cbfa7 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:43.443723 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5952d3-81d2-44fe-b91b-1a4bae7cbfa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:43.444001 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5952d3-81d2-44fe-b91b-1a4bae7cbfa7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:43.452833 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-eca237b2-ac2e-4bfb-98eb-f440a9df69d1 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:43.453366 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-eca237b2-ac2e-4bfb-98eb-f440a9df69d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:43.529470 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5952d3-81d2-44fe-b91b-1a4bae7cbfa7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:43.530818 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 253/496] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:27:43.530993 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5952d3-81d2-44fe-b91b-1a4bae7cbfa7 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:43.532018 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca237b2-ac2e-4bfb-98eb-f440a9df69d1 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:43.646861 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca237b2-ac2e-4bfb-98eb-f440a9df69d1 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:43.647048 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 244/497] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:27:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 08:27:43.648179 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc025e9-9545-4d8c-b50f-71b162a0a818 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:43.648543 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc025e9-9545-4d8c-b50f-71b162a0a818 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:43.648861 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc025e9-9545-4d8c-b50f-71b162a0a818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:43.669344 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:43.673401 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-7cc025e9-9545-4d8c-b50f-71b162a0a818 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:43.673989 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-7cc025e9-9545-4d8c-b50f-71b162a0a818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:43.691271 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53409062-2ba4-44aa-a4c5-ee83d0f050ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:43.691943 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53409062-2ba4-44aa-a4c5-ee83d0f050ee None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:43.692232 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53409062-2ba4-44aa-a4c5-ee83d0f050ee None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:43.692457 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:43.692558 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53409062-2ba4-44aa-a4c5-ee83d0f050ee None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:43.693227 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53409062-2ba4-44aa-a4c5-ee83d0f050ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:43.693539 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53409062-2ba4-44aa-a4c5-ee83d0f050ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:43.695025 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53409062-2ba4-44aa-a4c5-ee83d0f050ee None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:43.709512 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 254/498] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:27:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:43.710907 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53409062-2ba4-44aa-a4c5-ee83d0f050ee None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:43.762066 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 245/499] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:27:43.762986 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc025e9-9545-4d8c-b50f-71b162a0a818 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:43.830260 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53409062-2ba4-44aa-a4c5-ee83d0f050ee None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:43.864787 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffab6089-71e3-4e0a-b42a-30b8a9ae6962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:43.865652 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab6089-71e3-4e0a-b42a-30b8a9ae6962 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:43.865966 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab6089-71e3-4e0a-b42a-30b8a9ae6962 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:43.866236 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab6089-71e3-4e0a-b42a-30b8a9ae6962 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:43.866338 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc025e9-9545-4d8c-b50f-71b162a0a818 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:43.877633 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffab6089-71e3-4e0a-b42a-30b8a9ae6962 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:27:43.886001 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-ffab6089-71e3-4e0a-b42a-30b8a9ae6962 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:27:43.886001 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:43.886001 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:43.886001 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:43.886001 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:43.886001 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:43.886001 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:43.886001 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:43.886001 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:43.886001 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:43.886001 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:43.886001 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:43.886001 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:27:43.886001 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:27:43.886001 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:27:43.886001 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:27:43.887108 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:27:43.887108 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:27:43.887108 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:27:43.887108 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:27:43.887108 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:43.887108 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:43.887108 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:43.887108 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:43.887108 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:43.887108 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:43.887108 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:43.887108 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:43.887108 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:43.887108 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:43.887108 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:43.887108 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:43.887941 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:43.887941 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:43.887941 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:43.887941 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:43.887941 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:27:43.887941 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:27:43.887941 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:27:43.887941 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:27:43.887941 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:27:43.887941 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:27:43.887941 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:27:43.887941 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:43.887941 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 255/500] 10.222.0.13 () {56 vars in 1102 bytes} [Fri Jun 24 08:27:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:43.888600 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffab6089-71e3-4e0a-b42a-30b8a9ae6962 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:43.907723 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1513b4b6-0f3d-46b0-83a0-cbc7f99e1fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:43.908440 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1513b4b6-0f3d-46b0-83a0-cbc7f99e1fec None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:43.908725 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1513b4b6-0f3d-46b0-83a0-cbc7f99e1fec None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:43.909005 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1513b4b6-0f3d-46b0-83a0-cbc7f99e1fec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:43.920023 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1513b4b6-0f3d-46b0-83a0-cbc7f99e1fec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:43.921479 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1513b4b6-0f3d-46b0-83a0-cbc7f99e1fec None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:43.923652 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:43.929788 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:43.941230 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-1513b4b6-0f3d-46b0-83a0-cbc7f99e1fec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:43.941230 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:43.941230 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:43.941230 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:43.941230 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:43.941230 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:43.941230 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:43.941230 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:43.941230 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:43.941230 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:43.941230 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:43.941230 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:43.941230 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:27:43.941230 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:27:43.941230 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:27:43.942141 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:27:43.942141 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:43.942141 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:43.942141 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:43.942141 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:43.942141 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:43.942141 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:43.942141 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:43.942141 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:43.942141 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:43.942141 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:43.942141 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:43.942141 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:43.942141 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:43.942141 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:43.942141 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:43.942141 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:43.948649 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:27:43.948649 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:27:43.948649 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:43.948649 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:27:43.948649 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 246/501] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:27:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:43.948649 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1513b4b6-0f3d-46b0-83a0-cbc7f99e1fec None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:43.992647 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffab6089-71e3-4e0a-b42a-30b8a9ae6962 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:44.027171 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e990a8-e251-40b9-bd99-01c74d377d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:44.027963 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e990a8-e251-40b9-bd99-01c74d377d33 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:44.028248 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e990a8-e251-40b9-bd99-01c74d377d33 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:44.028524 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e990a8-e251-40b9-bd99-01c74d377d33 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:44.036057 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e990a8-e251-40b9-bd99-01c74d377d33 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:44.037504 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e990a8-e251-40b9-bd99-01c74d377d33 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:44.043590 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1513b4b6-0f3d-46b0-83a0-cbc7f99e1fec None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:44.052865 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-65e990a8-e251-40b9-bd99-01c74d377d33 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 24 08:27:44.052865 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:44.052865 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:44.052865 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:44.052865 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:44.052865 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:44.052865 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:44.052865 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:44.052865 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:44.052865 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:44.052865 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:44.052865 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:44.052865 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:27:44.052865 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:27:44.052865 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:27:44.052865 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:27:44.053881 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:44.053881 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:44.053881 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:27:44.053881 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:27:44.053881 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:27:44.053881 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:27:44.053881 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:44.053881 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:44.053881 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:44.053881 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:44.053881 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:44.053881 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:44.053881 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:44.053881 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:44.053881 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:44.053881 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:44.053881 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:44.053881 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:44.054754 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:44.054754 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:44.054754 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:27:44.054754 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:27:44.054754 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:27:44.054754 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:27:44.054754 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:27:44.054754 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:27:44.054754 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 24 08:27:44.054754 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:44.054754 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 256/502] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:27:43 2022] GET /identity/v3/users/glance => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:44.055931 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6893de-d6a4-4c56-8f54-dc5b51148eff None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:44.055931 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6893de-d6a4-4c56-8f54-dc5b51148eff None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:44.055931 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6893de-d6a4-4c56-8f54-dc5b51148eff None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:44.057131 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 257/503] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:44.057826 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6893de-d6a4-4c56-8f54-dc5b51148eff None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:44.083553 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9b3163-7f73-4ea5-be16-ff00f9796bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:44.084355 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b3163-7f73-4ea5-be16-ff00f9796bb4 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:44.084645 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b3163-7f73-4ea5-be16-ff00f9796bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:44.084925 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b3163-7f73-4ea5-be16-ff00f9796bb4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:44.095638 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9b3163-7f73-4ea5-be16-ff00f9796bb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:27:44.103935 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-4e9b3163-7f73-4ea5-be16-ff00f9796bb4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:27:44.103935 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:44.103935 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:44.103935 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:44.103935 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:44.103935 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:44.103935 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:44.103935 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:44.103935 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:44.103935 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:44.103935 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:44.103935 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:44.103935 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:27:44.103935 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:27:44.103935 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:27:44.103935 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:27:44.104996 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:27:44.104996 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:27:44.104996 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:27:44.104996 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:27:44.104996 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:44.104996 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:44.104996 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:44.104996 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:44.104996 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:44.104996 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:44.104996 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:44.104996 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:44.104996 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:44.104996 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:44.104996 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:44.104996 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:44.105833 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:44.105833 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:44.105833 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:44.105833 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:44.105833 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:27:44.105833 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:27:44.105833 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:27:44.105833 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:27:44.105833 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:27:44.105833 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:27:44.105833 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:27:44.105833 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:27:44.105833 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 247/504] 10.222.0.13 () {56 vars in 1102 bytes} [Fri Jun 24 08:27:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:44.106528 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9b3163-7f73-4ea5-be16-ff00f9796bb4 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:44.161560 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:44.166333 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:44.169617 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6893de-d6a4-4c56-8f54-dc5b51148eff None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:44.205320 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d948e8-42d0-4037-b0cc-b038349c2702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:44.206029 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d948e8-42d0-4037-b0cc-b038349c2702 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:44.206324 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d948e8-42d0-4037-b0cc-b038349c2702 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:44.206598 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d948e8-42d0-4037-b0cc-b038349c2702 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:44.207380 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d948e8-42d0-4037-b0cc-b038349c2702 None admin] RBAC: Adding query filter params (name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:44.207689 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d948e8-42d0-4037-b0cc-b038349c2702 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:44.209088 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d948e8-42d0-4037-b0cc-b038349c2702 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:44.209959 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9b3163-7f73-4ea5-be16-ff00f9796bb4 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:44.218458 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 258/505] 10.222.0.13 () {56 vars in 1103 bytes} [Fri Jun 24 08:27:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:44.219320 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d948e8-42d0-4037-b0cc-b038349c2702 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:44.246070 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a21079-d5a0-40dd-ba9f-51fc86c9de54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:44.246742 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a21079-d5a0-40dd-ba9f-51fc86c9de54 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:44.247028 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a21079-d5a0-40dd-ba9f-51fc86c9de54 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:44.247308 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a21079-d5a0-40dd-ba9f-51fc86c9de54 None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:44.248039 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a21079-d5a0-40dd-ba9f-51fc86c9de54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:44.248348 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a21079-d5a0-40dd-ba9f-51fc86c9de54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:44.249761 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a21079-d5a0-40dd-ba9f-51fc86c9de54 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:44.262092 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 248/506] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:27:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:44.263362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2cd0f5-c318-498a-a2bc-01d2cf77917c None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:44.263651 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2cd0f5-c318-498a-a2bc-01d2cf77917c None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:44.263927 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2cd0f5-c318-498a-a2bc-01d2cf77917c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:44.281032 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-7a2cd0f5-c318-498a-a2bc-01d2cf77917c None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:44.282286 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-7a2cd0f5-c318-498a-a2bc-01d2cf77917c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:44.321389 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d948e8-42d0-4037-b0cc-b038349c2702 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:44.349528 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 249/507] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:27:44.350558 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2cd0f5-c318-498a-a2bc-01d2cf77917c None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:44.359306 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c06c86-ca3d-4327-b686-996052a50cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:44.360127 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c06c86-ca3d-4327-b686-996052a50cb9 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:44.360544 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c06c86-ca3d-4327-b686-996052a50cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:44.361575 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c06c86-ca3d-4327-b686-996052a50cb9 None admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:44.361767 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c06c86-ca3d-4327-b686-996052a50cb9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:44.362068 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c06c86-ca3d-4327-b686-996052a50cb9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:44.363500 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c06c86-ca3d-4327-b686-996052a50cb9 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:44.403919 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 259/508] 10.222.0.13 () {56 vars in 1137 bytes} [Fri Jun 24 08:27:44 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:44.404780 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c06c86-ca3d-4327-b686-996052a50cb9 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:44.416059 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:44.429677 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:44.479438 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2cd0f5-c318-498a-a2bc-01d2cf77917c None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:44.514311 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c06c86-ca3d-4327-b686-996052a50cb9 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:44.518527 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaa39ed-3ad5-426d-8778-c67e8d5f9996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:44.519197 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa39ed-3ad5-426d-8778-c67e8d5f9996 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:44.519486 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa39ed-3ad5-426d-8778-c67e8d5f9996 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:44.519774 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa39ed-3ad5-426d-8778-c67e8d5f9996 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:44.520515 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaa39ed-3ad5-426d-8778-c67e8d5f9996 None admin] RBAC: Adding query filter params (name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:44.520825 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaa39ed-3ad5-426d-8778-c67e8d5f9996 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:44.522267 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaa39ed-3ad5-426d-8778-c67e8d5f9996 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:44.531509 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 250/509] 10.222.0.13 () {56 vars in 1103 bytes} [Fri Jun 24 08:27:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:44.532294 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaa39ed-3ad5-426d-8778-c67e8d5f9996 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:44.554676 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4593a51b-b98b-4186-973c-9cacb3cf1a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:44.555564 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4593a51b-b98b-4186-973c-9cacb3cf1a3b None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:44.555854 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4593a51b-b98b-4186-973c-9cacb3cf1a3b None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:44.556137 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4593a51b-b98b-4186-973c-9cacb3cf1a3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:44.566268 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4593a51b-b98b-4186-973c-9cacb3cf1a3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:44.567721 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4593a51b-b98b-4186-973c-9cacb3cf1a3b None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:44.587373 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-4593a51b-b98b-4186-973c-9cacb3cf1a3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:44.587373 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:44.587373 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:44.587373 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:44.587373 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:44.587373 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:44.587373 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:44.587373 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:44.587373 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:44.587373 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:44.587373 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:44.587373 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:44.587373 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:27:44.587373 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:27:44.587373 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:27:44.588219 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:27:44.588219 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:44.588219 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:44.588219 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:44.588219 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:44.588219 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:44.588219 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:44.588219 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:44.588219 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:44.588219 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:44.588219 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:44.588219 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:44.588219 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:44.588219 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:44.588219 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:44.588219 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:44.588219 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:44.589042 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:27:44.589042 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:27:44.589042 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:44.589042 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:44.589042 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 260/510] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:27:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:44.589351 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4593a51b-b98b-4186-973c-9cacb3cf1a3b None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:44.635514 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaa39ed-3ad5-426d-8778-c67e8d5f9996 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:44.648433 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:44.655536 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:44.669964 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c68563-b683-4022-af3a-78fed59b1365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:44.670760 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c68563-b683-4022-af3a-78fed59b1365 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:44.671047 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c68563-b683-4022-af3a-78fed59b1365 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:44.671326 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c68563-b683-4022-af3a-78fed59b1365 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:44.677909 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c68563-b683-4022-af3a-78fed59b1365 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:44.679357 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c68563-b683-4022-af3a-78fed59b1365 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:44.690015 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4593a51b-b98b-4186-973c-9cacb3cf1a3b None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:44.693513 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-76c68563-b683-4022-af3a-78fed59b1365 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:27:44.693513 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:44.693513 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:44.693513 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:44.693513 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:44.693513 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:44.693513 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:44.693513 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:44.693513 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:44.693513 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:44.693513 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:44.693513 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:44.693513 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:27:44.693513 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:27:44.693513 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:27:44.694447 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:27:44.694447 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:44.694447 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:44.694447 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:44.694447 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:44.694447 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:44.694447 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:44.694447 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:44.694447 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:44.694447 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:44.694447 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:44.694447 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:44.694447 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:44.694447 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:44.694447 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:44.694447 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:44.694447 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:44.695278 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:27:44.695278 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:27:44.695278 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:27:44.695278 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:27:44.695278 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:27:44.695278 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:27:44.695278 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:27:44.695278 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:27:44.695278 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 251/511] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:27:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:44.696227 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c68563-b683-4022-af3a-78fed59b1365 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:44.724800 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb808b2f-1094-4407-b212-c11b05b3848c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:44.725584 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb808b2f-1094-4407-b212-c11b05b3848c None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:44.725901 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb808b2f-1094-4407-b212-c11b05b3848c None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:44.726172 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb808b2f-1094-4407-b212-c11b05b3848c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:44.751460 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb808b2f-1094-4407-b212-c11b05b3848c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:44.753047 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb808b2f-1094-4407-b212-c11b05b3848c None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:44.774199 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-cb808b2f-1094-4407-b212-c11b05b3848c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:44.774199 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:44.774199 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:44.774199 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:44.774199 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:44.774199 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:44.774199 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:44.774199 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:44.774199 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:44.774199 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:44.774199 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:44.774199 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:44.774199 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:27:44.774199 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:27:44.774199 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:27:44.775575 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:27:44.775575 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:44.775575 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:44.775575 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:44.775575 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:44.775575 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:44.775575 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:44.775575 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:44.775575 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:44.775575 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:44.775575 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:44.775575 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:44.775575 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:44.775575 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:44.775575 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:44.775575 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:44.775575 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:44.776410 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:27:44.776410 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:27:44.776410 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:44.776410 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:44.776410 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 261/512] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:27:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:44.776410 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb808b2f-1094-4407-b212-c11b05b3848c None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:44.821395 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c68563-b683-4022-af3a-78fed59b1365 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:44.855982 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feebf895-5e83-4631-90af-61b9de7f1094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:44.856630 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feebf895-5e83-4631-90af-61b9de7f1094 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:44.856916 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feebf895-5e83-4631-90af-61b9de7f1094 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:44.857197 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feebf895-5e83-4631-90af-61b9de7f1094 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:44.875764 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb808b2f-1094-4407-b212-c11b05b3848c None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:44.897033 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:44.897204 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:44.911903 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b79aae1-95b2-4825-b5a7-4dd48416b485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:44.912680 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79aae1-95b2-4825-b5a7-4dd48416b485 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:44.912963 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79aae1-95b2-4825-b5a7-4dd48416b485 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:44.913232 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79aae1-95b2-4825-b5a7-4dd48416b485 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:44.924162 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feebf895-5e83-4631-90af-61b9de7f1094 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:44.925083 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b79aae1-95b2-4825-b5a7-4dd48416b485 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:44.925557 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feebf895-5e83-4631-90af-61b9de7f1094 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:44.926569 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b79aae1-95b2-4825-b5a7-4dd48416b485 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:44.944912 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-5b79aae1-95b2-4825-b5a7-4dd48416b485 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:44.944912 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:44.944912 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:44.944912 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:44.944912 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:44.944912 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:44.944912 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:44.944912 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:44.944912 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:44.944912 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:44.944912 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:44.944912 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:44.944912 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:27:44.944912 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:27:44.944912 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:27:44.945874 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:27:44.945874 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:44.945874 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:44.945874 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:44.945874 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:44.945874 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:44.945874 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:44.945874 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:44.945874 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:44.945874 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:44.945874 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:44.945874 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:44.945874 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:44.945874 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:44.945874 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:44.945874 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:44.945874 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:44.946722 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:27:44.946722 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:27:44.946722 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:44.946722 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:44.946722 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 262/513] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:27:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:44.946978 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b79aae1-95b2-4825-b5a7-4dd48416b485 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:45.024702 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 252/514] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:27:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:27:45.025478 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feebf895-5e83-4631-90af-61b9de7f1094 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:45.049451 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b79aae1-95b2-4825-b5a7-4dd48416b485 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:45.083022 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6dee01-6a5e-4482-87c5-417416a3f528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:45.083881 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6dee01-6a5e-4482-87c5-417416a3f528 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:45.084159 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6dee01-6a5e-4482-87c5-417416a3f528 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:45.084441 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6dee01-6a5e-4482-87c5-417416a3f528 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:45.085164 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6dee01-6a5e-4482-87c5-417416a3f528 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:45.085468 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6dee01-6a5e-4482-87c5-417416a3f528 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:45.087411 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6dee01-6a5e-4482-87c5-417416a3f528 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:45.093788 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:45.099170 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 263/515] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:27:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:45.099979 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6dee01-6a5e-4482-87c5-417416a3f528 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:45.131068 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feebf895-5e83-4631-90af-61b9de7f1094 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:45.132266 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:45.166332 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ef287-48e2-454a-9c3f-e614284fa984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:45.166990 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ef287-48e2-454a-9c3f-e614284fa984 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:45.167274 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ef287-48e2-454a-9c3f-e614284fa984 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:45.167552 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ef287-48e2-454a-9c3f-e614284fa984 None admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:45.168329 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76ef287-48e2-454a-9c3f-e614284fa984 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:45.168638 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76ef287-48e2-454a-9c3f-e614284fa984 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:45.170094 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76ef287-48e2-454a-9c3f-e614284fa984 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:45.187383 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 253/516] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Jun 24 08:27:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:45.188106 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ef287-48e2-454a-9c3f-e614284fa984 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:45.204434 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6dee01-6a5e-4482-87c5-417416a3f528 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:45.238501 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf60c19-e059-4ad5-b35a-a1651105efe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:45.239158 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf60c19-e059-4ad5-b35a-a1651105efe9 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:45.239443 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf60c19-e059-4ad5-b35a-a1651105efe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:45.239718 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf60c19-e059-4ad5-b35a-a1651105efe9 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:45.240433 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf60c19-e059-4ad5-b35a-a1651105efe9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:45.240736 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf60c19-e059-4ad5-b35a-a1651105efe9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:45.242152 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf60c19-e059-4ad5-b35a-a1651105efe9 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:45.255401 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 264/517] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:27:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:45.256221 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf60c19-e059-4ad5-b35a-a1651105efe9 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:45.287144 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ef287-48e2-454a-9c3f-e614284fa984 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:45.317879 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:45.323298 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5959d75-1806-4ec7-906f-fb66891b2b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:45.323957 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5959d75-1806-4ec7-906f-fb66891b2b27 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:45.324240 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5959d75-1806-4ec7-906f-fb66891b2b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:45.324520 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5959d75-1806-4ec7-906f-fb66891b2b27 None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:45.325229 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5959d75-1806-4ec7-906f-fb66891b2b27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:45.325532 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5959d75-1806-4ec7-906f-fb66891b2b27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:45.326949 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5959d75-1806-4ec7-906f-fb66891b2b27 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:45.337963 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 254/518] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:27:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:45.342948 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5959d75-1806-4ec7-906f-fb66891b2b27 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:45.347392 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:45.361357 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf60c19-e059-4ad5-b35a-a1651105efe9 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:45.396053 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc18f4-bc2e-45a4-9dcd-3141beec21c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:45.396845 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc18f4-bc2e-45a4-9dcd-3141beec21c5 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:45.397130 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc18f4-bc2e-45a4-9dcd-3141beec21c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:45.397405 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc18f4-bc2e-45a4-9dcd-3141beec21c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:45.409793 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bc18f4-bc2e-45a4-9dcd-3141beec21c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:45.411252 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bc18f4-bc2e-45a4-9dcd-3141beec21c5 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:45.429230 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-f8bc18f4-bc2e-45a4-9dcd-3141beec21c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:45.429230 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:45.429230 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:45.429230 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:45.429230 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:45.429230 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:45.429230 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:45.429230 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:45.429230 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:45.429230 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:45.429230 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:45.429230 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:45.429230 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:27:45.429230 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:27:45.429230 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:27:45.430899 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:27:45.430899 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:45.430899 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:45.430899 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:45.430899 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:45.430899 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:45.430899 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:45.430899 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:45.430899 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:45.430899 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:45.430899 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:45.430899 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:45.430899 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:45.430899 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:45.430899 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:45.430899 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:45.430899 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:45.431914 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:27:45.431914 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:27:45.431914 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:45.431914 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:45.435501 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 265/519] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:27:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:45.436369 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc18f4-bc2e-45a4-9dcd-3141beec21c5 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:45.445155 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5959d75-1806-4ec7-906f-fb66891b2b27 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:45.478311 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b24346-f506-4893-a048-18507b098aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:45.479098 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b24346-f506-4893-a048-18507b098aaf None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:45.479378 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b24346-f506-4893-a048-18507b098aaf None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:45.479647 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b24346-f506-4893-a048-18507b098aaf None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:45.487158 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b24346-f506-4893-a048-18507b098aaf None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:45.488594 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b24346-f506-4893-a048-18507b098aaf None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:45.503099 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-19b24346-f506-4893-a048-18507b098aaf None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 08:27:45.503099 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:45.503099 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:45.503099 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:45.503099 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:45.503099 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:45.503099 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:45.503099 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:45.503099 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:45.503099 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:45.503099 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:45.503099 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:45.503099 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:27:45.503099 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:27:45.503099 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:27:45.503099 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:27:45.504122 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:45.504122 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:45.504122 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:27:45.504122 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:27:45.504122 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:27:45.504122 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:27:45.504122 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:45.504122 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:45.504122 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:45.504122 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:45.504122 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:45.504122 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:45.504122 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:45.504122 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:45.504122 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:45.504122 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:45.504122 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:45.504122 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:45.505185 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:45.505185 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:45.505185 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:27:45.505185 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:27:45.505185 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:27:45.505185 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:27:45.505185 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:27:45.505185 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:27:45.505185 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 08:27:45.505185 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:27:45.505185 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 255/520] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:27:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:45.505185 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b24346-f506-4893-a048-18507b098aaf None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:45.540386 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc18f4-bc2e-45a4-9dcd-3141beec21c5 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:45.558670 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:45.576372 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f744bd0-e1c8-4263-9e61-c35a77649568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:45.577055 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f744bd0-e1c8-4263-9e61-c35a77649568 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:45.577331 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f744bd0-e1c8-4263-9e61-c35a77649568 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:45.577603 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f744bd0-e1c8-4263-9e61-c35a77649568 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:45.578447 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f744bd0-e1c8-4263-9e61-c35a77649568 None admin] RBAC: Adding query filter params (role.id=6ca1ae4292d9414ca6526c6e8608de9e, scope.project.id=180b0af5863349e99ae412b07c1e9465, user.id=74bb1bc6821b4500b26f160ccae5b7d5) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:45.578753 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f744bd0-e1c8-4263-9e61-c35a77649568 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:45.580163 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f744bd0-e1c8-4263-9e61-c35a77649568 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:45.585504 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 266/521] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Jun 24 08:27:45 2022] GET /identity/v3/role_assignments?user.id=74bb1bc6821b4500b26f160ccae5b7d5&scope.project.id=180b0af5863349e99ae412b07c1e9465&role.id=6ca1ae4292d9414ca6526c6e8608de9e => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:45.586403 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f744bd0-e1c8-4263-9e61-c35a77649568 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:45.607842 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:45.611982 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b24346-f506-4893-a048-18507b098aaf None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:45.654703 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bb175f-d50c-47b0-9983-fdafd4e9ef3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:45.655382 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb175f-d50c-47b0-9983-fdafd4e9ef3f None admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:45.655668 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb175f-d50c-47b0-9983-fdafd4e9ef3f None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:45.655946 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb175f-d50c-47b0-9983-fdafd4e9ef3f None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:45.656658 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bb175f-d50c-47b0-9983-fdafd4e9ef3f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:45.658171 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bb175f-d50c-47b0-9983-fdafd4e9ef3f None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:45.680419 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 256/522] 10.222.0.13 () {60 vars in 1132 bytes} [Fri Jun 24 08:27:45 2022] POST /identity/v3/users => generated 247 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:27:45.681198 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bb175f-d50c-47b0-9983-fdafd4e9ef3f None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:45.735956 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f744bd0-e1c8-4263-9e61-c35a77649568 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:45.786678 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf44e4c-2b74-4323-8652-a471a5d76ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:45.787635 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf44e4c-2b74-4323-8652-a471a5d76ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:45.787985 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf44e4c-2b74-4323-8652-a471a5d76ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:45.788332 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf44e4c-2b74-4323-8652-a471a5d76ac7 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:45.798323 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf44e4c-2b74-4323-8652-a471a5d76ac7 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:45.800040 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf44e4c-2b74-4323-8652-a471a5d76ac7 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:45.817505 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-ebf44e4c-2b74-4323-8652-a471a5d76ac7 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 08:27:45.817505 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:45.817505 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:45.817505 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:45.817505 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:45.817505 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:45.817505 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:45.817505 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:45.817505 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:45.817505 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:45.817505 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:45.817505 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:45.817505 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:27:45.817505 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:27:45.817505 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:27:45.817505 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:27:45.818802 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:45.818802 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:45.818802 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:27:45.818802 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:27:45.818802 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:27:45.818802 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:27:45.818802 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:45.818802 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:45.818802 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:45.818802 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:45.818802 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:45.818802 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:45.818802 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:45.818802 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:45.818802 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:45.818802 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:45.818802 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:45.818802 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:45.819884 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:45.819884 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:45.819884 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:27:45.819884 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:27:45.819884 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:27:45.819884 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:27:45.819884 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:27:45.819884 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:27:45.819884 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 08:27:45.819884 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:45.823921 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 267/523] 10.222.0.13 () {56 vars in 1102 bytes} [Fri Jun 24 08:27:45 2022] GET /identity/v3/users/neutron => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:45.824994 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf44e4c-2b74-4323-8652-a471a5d76ac7 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:45.828839 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bb175f-d50c-47b0-9983-fdafd4e9ef3f None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:45.841877 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:45.872814 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2836bc3-f313-4a9b-9070-ef0897aad71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:45.873478 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2836bc3-f313-4a9b-9070-ef0897aad71c None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:45.873802 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2836bc3-f313-4a9b-9070-ef0897aad71c None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:45.874244 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2836bc3-f313-4a9b-9070-ef0897aad71c None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:45.874973 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2836bc3-f313-4a9b-9070-ef0897aad71c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:45.875279 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2836bc3-f313-4a9b-9070-ef0897aad71c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:45.876924 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2836bc3-f313-4a9b-9070-ef0897aad71c None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:45.892196 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 257/524] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:27:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:45.893079 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2836bc3-f313-4a9b-9070-ef0897aad71c None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:45.915938 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:45.958654 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf44e4c-2b74-4323-8652-a471a5d76ac7 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:46.006475 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e12367-790e-4214-b4d0-caab5e12aae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:46.006984 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e12367-790e-4214-b4d0-caab5e12aae3 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:46.007273 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e12367-790e-4214-b4d0-caab5e12aae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:46.007546 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e12367-790e-4214-b4d0-caab5e12aae3 None admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:46.009244 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e12367-790e-4214-b4d0-caab5e12aae3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:46.009557 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e12367-790e-4214-b4d0-caab5e12aae3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:46.011011 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e12367-790e-4214-b4d0-caab5e12aae3 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:46.018356 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2836bc3-f313-4a9b-9070-ef0897aad71c None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:46.030535 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 268/525] 10.222.0.13 () {56 vars in 1137 bytes} [Fri Jun 24 08:27:45 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:46.031477 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e12367-790e-4214-b4d0-caab5e12aae3 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:46.054973 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84576199-f793-4221-95a3-b09716f3f80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:46.056539 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84576199-f793-4221-95a3-b09716f3f80e None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:46.057022 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84576199-f793-4221-95a3-b09716f3f80e None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:46.057491 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84576199-f793-4221-95a3-b09716f3f80e None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:46.058563 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84576199-f793-4221-95a3-b09716f3f80e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:46.059071 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84576199-f793-4221-95a3-b09716f3f80e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:46.060722 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84576199-f793-4221-95a3-b09716f3f80e None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:46.079555 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 258/526] 10.222.0.13 () {56 vars in 1139 bytes} [Fri Jun 24 08:27:45 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:46.080828 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84576199-f793-4221-95a3-b09716f3f80e None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:46.109842 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:46.146223 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e12367-790e-4214-b4d0-caab5e12aae3 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:46.174188 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:46.182927 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f579eb-ae9a-46da-8ca3-641b352c596f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:46.183724 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f579eb-ae9a-46da-8ca3-641b352c596f None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:46.184006 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f579eb-ae9a-46da-8ca3-641b352c596f None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:46.184279 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f579eb-ae9a-46da-8ca3-641b352c596f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:46.191631 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f579eb-ae9a-46da-8ca3-641b352c596f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:46.193085 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f579eb-ae9a-46da-8ca3-641b352c596f None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:46.200705 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84576199-f793-4221-95a3-b09716f3f80e None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:46.208210 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-85f579eb-ae9a-46da-8ca3-641b352c596f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:27:46.208210 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:46.208210 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:46.208210 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:46.208210 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:46.208210 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:46.208210 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:46.208210 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:46.208210 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:46.208210 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:46.208210 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:46.208210 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:46.208210 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:27:46.208210 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:27:46.208210 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:27:46.209089 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:27:46.209089 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:46.209089 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:46.209089 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:46.209089 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:46.209089 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:46.209089 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:46.209089 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:46.209089 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:46.209089 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:46.209089 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:46.209089 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:46.209089 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:46.209089 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:46.209089 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:46.209089 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:46.209089 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:46.209938 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:27:46.209938 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:27:46.209938 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:27:46.209938 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:27:46.209938 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:27:46.209938 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:27:46.209938 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:27:46.209938 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:46.213936 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 269/527] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:27:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:46.214780 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f579eb-ae9a-46da-8ca3-641b352c596f None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:46.246562 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9d7a6e-2a32-44d4-8985-8b67044b1806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:46.247593 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d7a6e-2a32-44d4-8985-8b67044b1806 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:46.248225 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d7a6e-2a32-44d4-8985-8b67044b1806 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:46.248790 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d7a6e-2a32-44d4-8985-8b67044b1806 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:46.322704 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f579eb-ae9a-46da-8ca3-641b352c596f None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:46.326771 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9d7a6e-2a32-44d4-8985-8b67044b1806 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:46.328476 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9d7a6e-2a32-44d4-8985-8b67044b1806 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:46.360439 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a12592a-a459-4a6b-ba43-e960897faf03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:46.361092 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a12592a-a459-4a6b-ba43-e960897faf03 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:46.361374 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a12592a-a459-4a6b-ba43-e960897faf03 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:46.361649 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a12592a-a459-4a6b-ba43-e960897faf03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:46.365901 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:46.421791 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:46.435683 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a12592a-a459-4a6b-ba43-e960897faf03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:46.437142 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a12592a-a459-4a6b-ba43-e960897faf03 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:46.454313 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 259/528] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:27:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:27:46.455552 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9d7a6e-2a32-44d4-8985-8b67044b1806 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:46.540794 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 270/529] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:27:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:27:46.541512 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a12592a-a459-4a6b-ba43-e960897faf03 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:46.566660 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9d7a6e-2a32-44d4-8985-8b67044b1806 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:46.596165 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:46.607503 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e250328-3c88-40da-a579-93c5073d6eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:46.608428 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e250328-3c88-40da-a579-93c5073d6eb9 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:46.609002 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e250328-3c88-40da-a579-93c5073d6eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:46.609467 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e250328-3c88-40da-a579-93c5073d6eb9 None admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:46.610694 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e250328-3c88-40da-a579-93c5073d6eb9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:46.611264 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e250328-3c88-40da-a579-93c5073d6eb9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:46.612992 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e250328-3c88-40da-a579-93c5073d6eb9 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:46.626747 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 260/530] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Jun 24 08:27:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:46.627886 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e250328-3c88-40da-a579-93c5073d6eb9 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:46.636034 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:46.656940 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a12592a-a459-4a6b-ba43-e960897faf03 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:46.706918 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcffe29c-454b-441b-a92d-6df26d54b6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:46.708035 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcffe29c-454b-441b-a92d-6df26d54b6fc None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:46.708557 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcffe29c-454b-441b-a92d-6df26d54b6fc None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:46.709226 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcffe29c-454b-441b-a92d-6df26d54b6fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:46.722991 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcffe29c-454b-441b-a92d-6df26d54b6fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:46.725637 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcffe29c-454b-441b-a92d-6df26d54b6fc None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:46.749908 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e250328-3c88-40da-a579-93c5073d6eb9 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:46.757907 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-fcffe29c-454b-441b-a92d-6df26d54b6fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:46.757907 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:46.757907 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:46.757907 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:46.757907 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:46.757907 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:46.757907 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:46.757907 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:46.757907 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:46.757907 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:46.757907 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:46.757907 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:46.757907 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:27:46.757907 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:27:46.757907 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:27:46.759567 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:27:46.759567 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:46.759567 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:46.759567 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:46.759567 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:46.759567 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:46.759567 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:46.759567 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:46.759567 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:46.759567 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:46.759567 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:46.759567 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:46.759567 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:46.759567 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:46.759567 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:46.759567 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:46.759567 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:46.760390 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:27:46.760390 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:27:46.760390 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:46.760390 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:46.763525 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 271/531] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:27:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:46.764570 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcffe29c-454b-441b-a92d-6df26d54b6fc None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:46.794598 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9a3122-cdd0-4d95-999d-0c118a4aa462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:46.795545 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a3122-cdd0-4d95-999d-0c118a4aa462 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:46.795929 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a3122-cdd0-4d95-999d-0c118a4aa462 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:46.796301 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a3122-cdd0-4d95-999d-0c118a4aa462 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:46.809163 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9a3122-cdd0-4d95-999d-0c118a4aa462 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:46.810870 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9a3122-cdd0-4d95-999d-0c118a4aa462 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:46.831438 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-0f9a3122-cdd0-4d95-999d-0c118a4aa462 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:46.831438 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:46.831438 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:46.831438 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:46.831438 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:46.831438 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:46.831438 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:46.831438 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:46.831438 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:46.831438 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:46.831438 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:46.831438 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:46.831438 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:27:46.831438 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:27:46.831438 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:27:46.832748 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:27:46.832748 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:46.832748 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:46.832748 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:46.832748 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:46.832748 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:46.832748 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:46.832748 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:46.832748 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:46.832748 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:46.832748 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:46.832748 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:46.832748 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:46.832748 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:46.832748 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:46.832748 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:46.832748 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:46.833587 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:27:46.833587 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:27:46.833587 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:46.833587 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:27:46.838238 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 261/532] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:27:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:46.839247 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9a3122-cdd0-4d95-999d-0c118a4aa462 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:46.886220 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:46.899645 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcffe29c-454b-441b-a92d-6df26d54b6fc None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:46.949218 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d525ee2b-79d5-464b-b7ec-95aee4866e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:46.950439 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525ee2b-79d5-464b-b7ec-95aee4866e83 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:46.950725 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525ee2b-79d5-464b-b7ec-95aee4866e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:46.951273 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525ee2b-79d5-464b-b7ec-95aee4866e83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:46.952128 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d525ee2b-79d5-464b-b7ec-95aee4866e83 None admin] RBAC: Adding query filter params (role.id=6ca1ae4292d9414ca6526c6e8608de9e, scope.project.id=180b0af5863349e99ae412b07c1e9465, user.id=d15c5f6aeb874191b73c8ee0410c6bc1) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:46.952689 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d525ee2b-79d5-464b-b7ec-95aee4866e83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:46.954833 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d525ee2b-79d5-464b-b7ec-95aee4866e83 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:46.959396 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9a3122-cdd0-4d95-999d-0c118a4aa462 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:46.965785 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 272/533] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Jun 24 08:27:46 2022] GET /identity/v3/role_assignments?user.id=d15c5f6aeb874191b73c8ee0410c6bc1&scope.project.id=180b0af5863349e99ae412b07c1e9465&role.id=6ca1ae4292d9414ca6526c6e8608de9e => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:46.966704 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d525ee2b-79d5-464b-b7ec-95aee4866e83 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:46.977782 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:47.042487 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d306b-119f-4c13-834f-7feab122d9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:47.043173 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d306b-119f-4c13-834f-7feab122d9cd None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:47.043495 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d306b-119f-4c13-834f-7feab122d9cd None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:47.043774 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d306b-119f-4c13-834f-7feab122d9cd None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:47.044506 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85d306b-119f-4c13-834f-7feab122d9cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:47.044812 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85d306b-119f-4c13-834f-7feab122d9cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:47.046247 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85d306b-119f-4c13-834f-7feab122d9cd None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:47.073900 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 262/534] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:27:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:47.079898 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d306b-119f-4c13-834f-7feab122d9cd None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:47.171846 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d525ee2b-79d5-464b-b7ec-95aee4866e83 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:47.213325 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef5c39-68c8-4a7a-bada-256632cce374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:47.214775 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef5c39-68c8-4a7a-bada-256632cce374 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:47.215016 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef5c39-68c8-4a7a-bada-256632cce374 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:47.215478 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef5c39-68c8-4a7a-bada-256632cce374 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:47.217772 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bef5c39-68c8-4a7a-bada-256632cce374 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:47.218128 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bef5c39-68c8-4a7a-bada-256632cce374 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:47.219620 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bef5c39-68c8-4a7a-bada-256632cce374 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:47.231687 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:47.236636 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 273/535] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:27:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:47.242925 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef5c39-68c8-4a7a-bada-256632cce374 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:47.248181 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d306b-119f-4c13-834f-7feab122d9cd None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:47.299108 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dc6dd6-61e8-4d2c-95fd-1d772f49f085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:47.300234 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc6dd6-61e8-4d2c-95fd-1d772f49f085 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:47.300673 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc6dd6-61e8-4d2c-95fd-1d772f49f085 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:47.301105 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc6dd6-61e8-4d2c-95fd-1d772f49f085 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:47.308638 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dc6dd6-61e8-4d2c-95fd-1d772f49f085 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:47.310427 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dc6dd6-61e8-4d2c-95fd-1d772f49f085 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:47.325723 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-e5dc6dd6-61e8-4d2c-95fd-1d772f49f085 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:27:47.325723 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:47.325723 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:47.325723 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:47.325723 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:47.325723 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:47.325723 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:47.325723 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:47.325723 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:47.325723 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:47.325723 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:47.325723 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:47.325723 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:27:47.325723 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:27:47.325723 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:27:47.326757 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:27:47.326757 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:47.326757 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:47.326757 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:47.326757 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:47.326757 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:47.326757 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:47.326757 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:47.326757 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:47.326757 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:47.326757 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:47.326757 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:47.326757 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:47.326757 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:47.326757 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:47.326757 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:47.326757 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:47.327788 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:27:47.327788 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:27:47.327788 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:27:47.327788 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:27:47.327788 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:27:47.327788 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:27:47.327788 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:27:47.327788 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:27:47.330028 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 263/536] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:27:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:47.336017 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dc6dd6-61e8-4d2c-95fd-1d772f49f085 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:47.380455 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:47.455013 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef5c39-68c8-4a7a-bada-256632cce374 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:47.526444 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e24b4d-748a-46ae-9f66-16761f828423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:47.527577 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e24b4d-748a-46ae-9f66-16761f828423 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:47.528008 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e24b4d-748a-46ae-9f66-16761f828423 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:47.528426 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e24b4d-748a-46ae-9f66-16761f828423 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:47.533081 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dc6dd6-61e8-4d2c-95fd-1d772f49f085 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:47.540070 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e24b4d-748a-46ae-9f66-16761f828423 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:47.541861 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e24b4d-748a-46ae-9f66-16761f828423 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:47.562153 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-e9e24b4d-748a-46ae-9f66-16761f828423 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:27:47.562153 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:47.562153 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:47.562153 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:47.562153 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:47.562153 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:47.562153 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:47.562153 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:47.562153 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:47.562153 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:47.562153 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:47.562153 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:47.562153 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:27:47.562153 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:27:47.562153 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:27:47.563048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:27:47.563048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:47.563048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:47.563048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:47.563048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:47.563048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:47.563048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:47.563048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:47.563048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:47.563048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:47.563048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:47.563048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:47.563048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:47.563048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:47.563048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:47.563048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:47.563048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:47.563901 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:27:47.563901 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:27:47.563901 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:27:47.563901 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:27:47.563901 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:27:47.563901 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:27:47.563901 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:27:47.563901 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:47.566159 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 274/537] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:27:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:47.572491 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e24b4d-748a-46ae-9f66-16761f828423 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:47.603565 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bbd50f-d483-4a5c-a37d-aecc6ab4132d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:47.604377 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bbd50f-d483-4a5c-a37d-aecc6ab4132d None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:47.604758 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bbd50f-d483-4a5c-a37d-aecc6ab4132d None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:47.605150 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bbd50f-d483-4a5c-a37d-aecc6ab4132d None admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:47.606063 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bbd50f-d483-4a5c-a37d-aecc6ab4132d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:47.606463 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bbd50f-d483-4a5c-a37d-aecc6ab4132d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:47.608147 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bbd50f-d483-4a5c-a37d-aecc6ab4132d None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:47.629576 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 264/538] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Jun 24 08:27:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:47.635039 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bbd50f-d483-4a5c-a37d-aecc6ab4132d None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:47.671681 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:47.708352 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e24b4d-748a-46ae-9f66-16761f828423 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:47.754100 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:47.779346 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8195fc-c3ae-4f8e-9698-dfb5a7dca8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:47.780009 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8195fc-c3ae-4f8e-9698-dfb5a7dca8ff None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:47.780291 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8195fc-c3ae-4f8e-9698-dfb5a7dca8ff None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:47.780563 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8195fc-c3ae-4f8e-9698-dfb5a7dca8ff None admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:47.781347 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8195fc-c3ae-4f8e-9698-dfb5a7dca8ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:47.781652 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8195fc-c3ae-4f8e-9698-dfb5a7dca8ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:47.783101 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8195fc-c3ae-4f8e-9698-dfb5a7dca8ff None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:47.810491 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 275/539] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Jun 24 08:27:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:47.811877 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bbd50f-d483-4a5c-a37d-aecc6ab4132d None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:47.811993 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad4228-228d-412f-8486-22639f2a18b6 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:47.812171 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad4228-228d-412f-8486-22639f2a18b6 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:47.812431 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad4228-228d-412f-8486-22639f2a18b6 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:47.813866 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 276/540] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:47.819302 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad4228-228d-412f-8486-22639f2a18b6 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:47.853015 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b2b86d-05ad-4b07-a677-266b9bc70538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:47.853757 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2b86d-05ad-4b07-a677-266b9bc70538 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:47.854048 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2b86d-05ad-4b07-a677-266b9bc70538 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:47.854429 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2b86d-05ad-4b07-a677-266b9bc70538 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:47.855368 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b2b86d-05ad-4b07-a677-266b9bc70538 None admin] RBAC: Adding query filter params (role.id=6ca1ae4292d9414ca6526c6e8608de9e, scope.project.id=180b0af5863349e99ae412b07c1e9465, user.id=44dcdfb5ca1541f9b4967a1f8ddbb34b) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:47.855675 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b2b86d-05ad-4b07-a677-266b9bc70538 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:47.857213 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b2b86d-05ad-4b07-a677-266b9bc70538 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:47.863952 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 265/541] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Jun 24 08:27:47 2022] GET /identity/v3/role_assignments?user.id=44dcdfb5ca1541f9b4967a1f8ddbb34b&scope.project.id=180b0af5863349e99ae412b07c1e9465&role.id=6ca1ae4292d9414ca6526c6e8608de9e => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:47.865204 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40255775-978e-46d5-bd1f-8a22bb37d25a None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:47.865491 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40255775-978e-46d5-bd1f-8a22bb37d25a None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:47.874120 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40255775-978e-46d5-bd1f-8a22bb37d25a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:47.907353 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-40255775-978e-46d5-bd1f-8a22bb37d25a None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:47.908573 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-40255775-978e-46d5-bd1f-8a22bb37d25a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:47.953268 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad4228-228d-412f-8486-22639f2a18b6 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:47.996854 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02a0674-17f3-4a37-8037-67d5b4cfcd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:47.997655 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a0674-17f3-4a37-8037-67d5b4cfcd48 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:47.997958 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a0674-17f3-4a37-8037-67d5b4cfcd48 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:47.998230 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a0674-17f3-4a37-8037-67d5b4cfcd48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:47.999073 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02a0674-17f3-4a37-8037-67d5b4cfcd48 None admin] RBAC: Adding query filter params (role.id=6ca1ae4292d9414ca6526c6e8608de9e, scope.project.id=180b0af5863349e99ae412b07c1e9465, user.id=c5f7ba6c7faf43e7acbb43eee58d8c8d) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:47.999381 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02a0674-17f3-4a37-8037-67d5b4cfcd48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:48.001079 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02a0674-17f3-4a37-8037-67d5b4cfcd48 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:48.008976 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 277/542] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Jun 24 08:27:47 2022] GET /identity/v3/role_assignments?user.id=c5f7ba6c7faf43e7acbb43eee58d8c8d&scope.project.id=180b0af5863349e99ae412b07c1e9465&role.id=6ca1ae4292d9414ca6526c6e8608de9e => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:48.022020 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d456aa8b-6ac3-4967-abcd-9303036a7c50 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:48.022410 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d456aa8b-6ac3-4967-abcd-9303036a7c50 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:48.022656 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d456aa8b-6ac3-4967-abcd-9303036a7c50 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:48.029505 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 278/543] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:27:48.031395 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2329ff7-e45d-42aa-ad9c-5e424a0cbf99 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:48.031688 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2329ff7-e45d-42aa-ad9c-5e424a0cbf99 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:48.031967 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2329ff7-e45d-42aa-ad9c-5e424a0cbf99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:48.056560 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-d2329ff7-e45d-42aa-ad9c-5e424a0cbf99 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:48.060363 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-d2329ff7-e45d-42aa-ad9c-5e424a0cbf99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:48.065311 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 266/544] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:27:48.115436 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40255775-978e-46d5-bd1f-8a22bb37d25a None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:48.121735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:48.144072 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:48.162692 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 279/545] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 08:27:48.185765 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2329ff7-e45d-42aa-ad9c-5e424a0cbf99 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:48.353509 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40255775-978e-46d5-bd1f-8a22bb37d25a None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:48.386057 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2329ff7-e45d-42aa-ad9c-5e424a0cbf99 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:48.437565 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a5d8f-bdab-463e-b128-903ea6ed85f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:48.438656 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a5d8f-bdab-463e-b128-903ea6ed85f1 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:48.438940 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a5d8f-bdab-463e-b128-903ea6ed85f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:48.439220 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a5d8f-bdab-463e-b128-903ea6ed85f1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:48.465299 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5a5d8f-bdab-463e-b128-903ea6ed85f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:27:48.476382 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62302ff8-b0b7-493c-a237-e7987f3baf6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:48.477355 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62302ff8-b0b7-493c-a237-e7987f3baf6d None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:48.477856 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62302ff8-b0b7-493c-a237-e7987f3baf6d None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:48.478145 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62302ff8-b0b7-493c-a237-e7987f3baf6d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:48.481585 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-cc5a5d8f-bdab-463e-b128-903ea6ed85f1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:27:48.481585 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:48.481585 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:48.481585 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:48.481585 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:48.481585 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:48.481585 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:48.481585 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:48.481585 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:48.481585 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:48.481585 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:48.481585 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:48.481585 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:27:48.481585 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:27:48.481585 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:27:48.481585 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:27:48.482484 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:27:48.482484 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:27:48.482484 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:27:48.482484 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:27:48.482484 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:48.482484 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:48.482484 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:48.482484 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:48.482484 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:48.482484 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:48.482484 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:48.482484 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:48.482484 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:48.482484 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:48.482484 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:48.482484 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:48.483294 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:48.483294 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:48.483294 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:48.483294 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:48.483294 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:27:48.483294 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:27:48.483294 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:27:48.483294 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:27:48.483294 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:27:48.483294 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:27:48.483294 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:27:48.483294 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:27:48.486888 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 267/546] 10.222.0.13 () {56 vars in 1102 bytes} [Fri Jun 24 08:27:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:27:48.492304 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a5d8f-bdab-463e-b128-903ea6ed85f1 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:48.501313 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62302ff8-b0b7-493c-a237-e7987f3baf6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:27:48.519636 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-62302ff8-b0b7-493c-a237-e7987f3baf6d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:27:48.519636 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:48.519636 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:48.519636 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:48.519636 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:48.519636 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:48.519636 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:48.519636 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:48.519636 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:48.519636 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:48.519636 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:48.519636 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:48.519636 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:27:48.519636 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:27:48.519636 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:27:48.519636 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:27:48.522580 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:27:48.522580 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:27:48.522580 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:27:48.522580 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:27:48.522580 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:48.522580 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:48.522580 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:48.522580 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:48.522580 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:48.522580 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:48.522580 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:48.522580 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:48.522580 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:48.522580 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:48.522580 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:48.522580 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:48.529813 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:48.529813 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:48.529813 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:48.529813 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:48.529813 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:27:48.529813 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:27:48.529813 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:27:48.529813 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:27:48.529813 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:27:48.529813 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:27:48.529813 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:27:48.529813 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:48.529813 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 280/547] 10.222.0.13 () {56 vars in 1102 bytes} [Fri Jun 24 08:27:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:48.529813 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62302ff8-b0b7-493c-a237-e7987f3baf6d None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:48.583291 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:48.639259 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a5d8f-bdab-463e-b128-903ea6ed85f1 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:48.658180 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:48.710428 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9117b422-5b60-4f20-919a-7c4137355f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:48.712206 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9117b422-5b60-4f20-919a-7c4137355f8e None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:48.712794 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9117b422-5b60-4f20-919a-7c4137355f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:48.713271 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9117b422-5b60-4f20-919a-7c4137355f8e None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:48.714318 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9117b422-5b60-4f20-919a-7c4137355f8e None admin] RBAC: Adding query filter params (name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:48.714827 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9117b422-5b60-4f20-919a-7c4137355f8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:48.716507 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9117b422-5b60-4f20-919a-7c4137355f8e None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:48.733233 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 268/548] 10.222.0.13 () {56 vars in 1103 bytes} [Fri Jun 24 08:27:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:27:48.744469 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62302ff8-b0b7-493c-a237-e7987f3baf6d None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:48.748410 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9117b422-5b60-4f20-919a-7c4137355f8e None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:48.787182 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e8acbb-ba02-4229-9a54-a6515ef2f06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:48.788651 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e8acbb-ba02-4229-9a54-a6515ef2f06a None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:48.788651 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e8acbb-ba02-4229-9a54-a6515ef2f06a None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:48.788651 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e8acbb-ba02-4229-9a54-a6515ef2f06a None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:48.789141 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e8acbb-ba02-4229-9a54-a6515ef2f06a None admin] RBAC: Adding query filter params (name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:48.789519 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e8acbb-ba02-4229-9a54-a6515ef2f06a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:48.791113 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e8acbb-ba02-4229-9a54-a6515ef2f06a None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:48.800463 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 281/549] 10.222.0.13 () {56 vars in 1103 bytes} [Fri Jun 24 08:27:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:27:48.808454 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e8acbb-ba02-4229-9a54-a6515ef2f06a None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:48.983687 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9117b422-5b60-4f20-919a-7c4137355f8e None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:49.032760 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e8acbb-ba02-4229-9a54-a6515ef2f06a None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:49.037468 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:49.059221 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b548ac0f-a0d9-4e04-98b8-18d3f93b4729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:49.060022 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548ac0f-a0d9-4e04-98b8-18d3f93b4729 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:49.060308 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548ac0f-a0d9-4e04-98b8-18d3f93b4729 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:49.060585 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548ac0f-a0d9-4e04-98b8-18d3f93b4729 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:49.068662 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:49.085864 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29107d66-4896-4ce3-b46f-0b39b55e5387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:49.086942 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29107d66-4896-4ce3-b46f-0b39b55e5387 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:49.087236 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29107d66-4896-4ce3-b46f-0b39b55e5387 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:49.087994 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29107d66-4896-4ce3-b46f-0b39b55e5387 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:49.089233 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b548ac0f-a0d9-4e04-98b8-18d3f93b4729 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:49.090732 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b548ac0f-a0d9-4e04-98b8-18d3f93b4729 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:49.099533 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29107d66-4896-4ce3-b46f-0b39b55e5387 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:49.101073 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29107d66-4896-4ce3-b46f-0b39b55e5387 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:49.121375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-29107d66-4896-4ce3-b46f-0b39b55e5387 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:49.121375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:49.121375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:49.121375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:49.121375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:49.121375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:49.121375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:49.121375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:49.121375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:49.121375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:49.121375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:49.121375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:49.121375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:27:49.121375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:27:49.121375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:27:49.122563 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:27:49.122563 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:49.122563 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:49.122563 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:49.122563 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:49.122563 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:49.122563 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:49.122563 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:49.122563 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:49.122563 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:49.122563 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:49.122563 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:49.122563 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:49.122563 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:49.122563 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:49.122563 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:49.122563 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:49.123624 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:27:49.123624 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:27:49.123624 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:49.123624 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:49.123624 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 282/550] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:27:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:49.130360 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-b548ac0f-a0d9-4e04-98b8-18d3f93b4729 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:49.130360 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:49.130360 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:49.130360 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:49.130360 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:49.130360 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:49.130360 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:49.130360 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:49.130360 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:49.130360 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:49.130360 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:49.130360 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:49.130360 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:27:49.130360 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:27:49.130360 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:27:49.131193 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:27:49.131193 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:49.131193 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:49.131193 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:49.131193 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:49.131193 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:49.131193 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:49.131193 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:49.131193 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:49.131193 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:49.131193 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:49.131193 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:49.131193 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:49.131193 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:49.131193 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:49.131193 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:49.131193 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:49.132618 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:27:49.132618 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:27:49.132618 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:49.132618 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:27:49.133407 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29107d66-4896-4ce3-b46f-0b39b55e5387 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:49.133552 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 269/551] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:27:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 384 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:27:49.137622 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b548ac0f-a0d9-4e04-98b8-18d3f93b4729 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:49.304317 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29107d66-4896-4ce3-b46f-0b39b55e5387 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:49.315715 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b548ac0f-a0d9-4e04-98b8-18d3f93b4729 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:49.359844 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c08ab-e737-4984-9121-30cab3a203db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:49.360518 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c08ab-e737-4984-9121-30cab3a203db None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:49.360800 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c08ab-e737-4984-9121-30cab3a203db None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:49.361073 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c08ab-e737-4984-9121-30cab3a203db None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:49.361835 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114c08ab-e737-4984-9121-30cab3a203db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:49.362142 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114c08ab-e737-4984-9121-30cab3a203db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:49.363557 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114c08ab-e737-4984-9121-30cab3a203db None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:49.389153 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 283/552] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:27:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:49.393741 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c08ab-e737-4984-9121-30cab3a203db None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:49.407782 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:49.411154 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ea184-70d1-4dbf-849e-118185c0ef17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:49.411898 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ea184-70d1-4dbf-849e-118185c0ef17 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:49.412178 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ea184-70d1-4dbf-849e-118185c0ef17 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:49.412450 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ea184-70d1-4dbf-849e-118185c0ef17 None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:49.413222 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748ea184-70d1-4dbf-849e-118185c0ef17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:49.413532 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748ea184-70d1-4dbf-849e-118185c0ef17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:49.415380 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748ea184-70d1-4dbf-849e-118185c0ef17 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:49.443974 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 270/553] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:27:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:49.449972 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ea184-70d1-4dbf-849e-118185c0ef17 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:49.514051 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:49.534524 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c08ab-e737-4984-9121-30cab3a203db None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:49.587598 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b81e50-6142-4803-8218-ea2e587abfa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:49.588340 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b81e50-6142-4803-8218-ea2e587abfa0 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:49.588628 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b81e50-6142-4803-8218-ea2e587abfa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:49.588907 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b81e50-6142-4803-8218-ea2e587abfa0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:49.603674 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b81e50-6142-4803-8218-ea2e587abfa0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:49.605135 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b81e50-6142-4803-8218-ea2e587abfa0 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:49.615345 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ea184-70d1-4dbf-849e-118185c0ef17 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:49.637436 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-95b81e50-6142-4803-8218-ea2e587abfa0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 08:27:49.637436 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:49.637436 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:49.637436 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:49.637436 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:49.637436 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:49.637436 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:49.637436 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:49.637436 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:49.637436 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:49.637436 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:49.637436 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:49.637436 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:27:49.637436 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:27:49.637436 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:27:49.637436 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:27:49.638483 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:49.638483 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:49.638483 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:27:49.638483 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:27:49.638483 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:27:49.638483 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:27:49.638483 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:49.638483 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:49.638483 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:49.638483 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:49.638483 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:49.638483 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:49.638483 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:49.638483 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:49.638483 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:49.638483 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:49.638483 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:49.638483 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:49.639379 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:49.639379 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:49.639379 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:27:49.639379 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:27:49.639379 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:27:49.639379 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:27:49.639379 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:27:49.639379 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:27:49.639379 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 08:27:49.639379 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:49.645056 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 284/554] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Jun 24 08:27:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:49.648327 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b81e50-6142-4803-8218-ea2e587abfa0 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:49.688512 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d9bfed-62b9-4d19-8efe-6587ea3eb3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:49.690376 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d9bfed-62b9-4d19-8efe-6587ea3eb3c7 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:49.690861 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d9bfed-62b9-4d19-8efe-6587ea3eb3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:49.691313 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d9bfed-62b9-4d19-8efe-6587ea3eb3c7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:49.704676 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d9bfed-62b9-4d19-8efe-6587ea3eb3c7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:49.707064 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d9bfed-62b9-4d19-8efe-6587ea3eb3c7 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:49.725889 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-a9d9bfed-62b9-4d19-8efe-6587ea3eb3c7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 08:27:49.725889 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:49.725889 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:49.725889 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:49.725889 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:49.725889 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:49.725889 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:49.725889 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:49.725889 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:49.725889 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:49.725889 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:49.725889 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:49.725889 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:27:49.725889 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:27:49.725889 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:27:49.725889 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:27:49.727010 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:49.727010 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:49.727010 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:27:49.727010 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:27:49.727010 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:27:49.727010 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:27:49.727010 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:49.727010 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:49.727010 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:49.727010 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:49.727010 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:49.727010 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:49.727010 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:49.727010 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:49.727010 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:49.727010 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:49.727010 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:49.727010 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:49.728966 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:49.728966 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:49.728966 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:27:49.728966 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:27:49.728966 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:27:49.728966 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:27:49.728966 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:27:49.728966 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:27:49.728966 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 08:27:49.728966 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:27:49.732741 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 271/555] 10.222.0.13 () {56 vars in 1093 bytes} [Fri Jun 24 08:27:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:27:49.734460 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd5a44-c489-4cc4-99fb-9b265e3a6525 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:49.734817 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd5a44-c489-4cc4-99fb-9b265e3a6525 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:49.735165 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd5a44-c489-4cc4-99fb-9b265e3a6525 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:49.741352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 272/556] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:49.741352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdd5a44-c489-4cc4-99fb-9b265e3a6525 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:49.753662 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:49.795009 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b81e50-6142-4803-8218-ea2e587abfa0 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:49.834968 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e24ef12-9fa5-43b2-95d9-8937b65b1df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:49.835964 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e24ef12-9fa5-43b2-95d9-8937b65b1df8 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:49.837245 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e24ef12-9fa5-43b2-95d9-8937b65b1df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:49.837928 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e24ef12-9fa5-43b2-95d9-8937b65b1df8 None admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:49.838410 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e24ef12-9fa5-43b2-95d9-8937b65b1df8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:49.838827 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e24ef12-9fa5-43b2-95d9-8937b65b1df8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:49.840359 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e24ef12-9fa5-43b2-95d9-8937b65b1df8 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:49.863568 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 285/557] 10.222.0.13 () {56 vars in 1135 bytes} [Fri Jun 24 08:27:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:49.863568 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df6b51-ad5d-4f25-be15-20e158a365fa None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:49.863568 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df6b51-ad5d-4f25-be15-20e158a365fa None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:49.863568 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df6b51-ad5d-4f25-be15-20e158a365fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:49.879687 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-71df6b51-ad5d-4f25-be15-20e158a365fa None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:49.881008 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-71df6b51-ad5d-4f25-be15-20e158a365fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:49.882294 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdd5a44-c489-4cc4-99fb-9b265e3a6525 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:49.894843 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:49.925793 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43814c69-8c84-4958-a838-76d54913a1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:49.926835 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43814c69-8c84-4958-a838-76d54913a1c1 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:49.926835 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43814c69-8c84-4958-a838-76d54913a1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:49.927031 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43814c69-8c84-4958-a838-76d54913a1c1 None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:49.927819 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43814c69-8c84-4958-a838-76d54913a1c1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:49.928129 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43814c69-8c84-4958-a838-76d54913a1c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:49.930227 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43814c69-8c84-4958-a838-76d54913a1c1 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:49.953173 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 273/558] 10.222.0.13 () {56 vars in 1133 bytes} [Fri Jun 24 08:27:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:49.953173 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43814c69-8c84-4958-a838-76d54913a1c1 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:49.969880 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 286/559] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:27:49.971149 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71df6b51-ad5d-4f25-be15-20e158a365fa None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:50.048436 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:50.096287 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43814c69-8c84-4958-a838-76d54913a1c1 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:50.107635 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71df6b51-ad5d-4f25-be15-20e158a365fa None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:50.135958 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b4811-24f7-45a3-8255-e1bd7dd5a807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:50.136897 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b4811-24f7-45a3-8255-e1bd7dd5a807 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:50.137417 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b4811-24f7-45a3-8255-e1bd7dd5a807 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:50.137905 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b4811-24f7-45a3-8255-e1bd7dd5a807 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:50.154781 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3823332b-dba0-4081-9236-1789a1d29072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:50.155878 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3823332b-dba0-4081-9236-1789a1d29072 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:50.156160 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3823332b-dba0-4081-9236-1789a1d29072 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:50.156438 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3823332b-dba0-4081-9236-1789a1d29072 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:50.171054 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3823332b-dba0-4081-9236-1789a1d29072 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:50.173378 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3823332b-dba0-4081-9236-1789a1d29072 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:50.178700 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:50.199001 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-3823332b-dba0-4081-9236-1789a1d29072 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:50.199001 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:50.199001 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:50.199001 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:50.199001 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:50.199001 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:50.199001 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:50.199001 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:50.199001 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:50.199001 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:50.199001 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:50.199001 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:50.199001 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:27:50.199001 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:27:50.199001 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:27:50.199932 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:27:50.199932 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:50.199932 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:50.199932 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:50.199932 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:50.199932 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:50.199932 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:50.199932 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:50.199932 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:50.199932 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:50.199932 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:50.199932 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:50.199932 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:50.199932 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:50.199932 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:50.199932 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:50.199932 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:50.200806 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:27:50.200806 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:27:50.200806 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:50.200806 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:50.200806 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 287/560] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:27:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:50.201070 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3823332b-dba0-4081-9236-1789a1d29072 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:50.243001 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464b4811-24f7-45a3-8255-e1bd7dd5a807 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:50.245922 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464b4811-24f7-45a3-8255-e1bd7dd5a807 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:50.313819 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3823332b-dba0-4081-9236-1789a1d29072 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:50.325673 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:50.354185 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d16580-c825-4998-9062-3a246c7ff53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:50.354902 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d16580-c825-4998-9062-3a246c7ff53e None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:50.355199 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d16580-c825-4998-9062-3a246c7ff53e None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:50.355457 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d16580-c825-4998-9062-3a246c7ff53e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:50.367792 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d16580-c825-4998-9062-3a246c7ff53e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:27:50.375518 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-12d16580-c825-4998-9062-3a246c7ff53e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:27:50.375518 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:50.375518 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:50.375518 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:50.375518 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:50.375518 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:50.375518 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:50.375518 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:50.375518 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:50.375518 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:50.375518 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:50.375518 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:50.375518 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:27:50.375518 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:27:50.375518 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:27:50.375518 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:27:50.376722 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:27:50.376722 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:27:50.376722 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:27:50.376722 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:27:50.376722 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:50.376722 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:50.376722 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:50.376722 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:50.376722 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:50.376722 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:50.376722 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:50.376722 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:50.376722 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:50.376722 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:50.376722 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:50.376722 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:50.377604 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:50.377604 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:50.377604 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:50.377604 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:50.377604 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:27:50.377604 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:27:50.377604 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:27:50.377604 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:27:50.377604 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:27:50.377604 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:27:50.377604 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:27:50.377604 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:50.381884 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 288/561] 10.222.0.13 () {56 vars in 1102 bytes} [Fri Jun 24 08:27:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:50.385789 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 274/562] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:27:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:27:50.385946 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d16580-c825-4998-9062-3a246c7ff53e None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:50.386026 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb55a85-ab48-4a55-905f-fb0bccd4bdcd None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:50.386026 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb55a85-ab48-4a55-905f-fb0bccd4bdcd None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:50.386026 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb55a85-ab48-4a55-905f-fb0bccd4bdcd None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:50.386399 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 275/563] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:50.387035 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb55a85-ab48-4a55-905f-fb0bccd4bdcd None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:50.460464 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:50.503163 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d16580-c825-4998-9062-3a246c7ff53e None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:50.510475 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb55a85-ab48-4a55-905f-fb0bccd4bdcd None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:50.536734 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d2a000-a3e5-4621-9369-4a8179abc873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:50.537393 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d2a000-a3e5-4621-9369-4a8179abc873 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:50.537672 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d2a000-a3e5-4621-9369-4a8179abc873 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:50.537981 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d2a000-a3e5-4621-9369-4a8179abc873 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:50.538691 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d2a000-a3e5-4621-9369-4a8179abc873 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:50.538992 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d2a000-a3e5-4621-9369-4a8179abc873 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:50.541018 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d2a000-a3e5-4621-9369-4a8179abc873 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:50.549016 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903ac71b-7abc-4d7a-a11a-a71038643db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:50.549675 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ac71b-7abc-4d7a-a11a-a71038643db3 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:50.549989 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ac71b-7abc-4d7a-a11a-a71038643db3 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:50.550654 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ac71b-7abc-4d7a-a11a-a71038643db3 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:50.551010 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903ac71b-7abc-4d7a-a11a-a71038643db3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:50.551318 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903ac71b-7abc-4d7a-a11a-a71038643db3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:50.552680 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903ac71b-7abc-4d7a-a11a-a71038643db3 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:50.553389 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 289/564] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:27:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:50.554203 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d2a000-a3e5-4621-9369-4a8179abc873 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:50.561534 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 276/565] 10.222.0.13 () {56 vars in 1103 bytes} [Fri Jun 24 08:27:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:50.562743 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67176e-99db-45e9-8c07-1bd46ff28d58 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:50.563102 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67176e-99db-45e9-8c07-1bd46ff28d58 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:50.563439 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67176e-99db-45e9-8c07-1bd46ff28d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:50.579765 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-4e67176e-99db-45e9-8c07-1bd46ff28d58 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:50.580957 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-4e67176e-99db-45e9-8c07-1bd46ff28d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:50.584225 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:50.647635 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 277/566] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:27:50.649382 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a7fbc-f117-4fad-8e67-27a9885f9193 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:50.649909 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a7fbc-f117-4fad-8e67-27a9885f9193 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:50.650189 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a7fbc-f117-4fad-8e67-27a9885f9193 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:50.651450 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 278/567] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:50.652195 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0a7fbc-f117-4fad-8e67-27a9885f9193 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:50.666127 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d2a000-a3e5-4621-9369-4a8179abc873 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:50.717291 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0af5fb7-57f5-48aa-bee7-fd9d6cdbf14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:50.718108 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0af5fb7-57f5-48aa-bee7-fd9d6cdbf14d None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:50.718373 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0af5fb7-57f5-48aa-bee7-fd9d6cdbf14d None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:50.718641 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0af5fb7-57f5-48aa-bee7-fd9d6cdbf14d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:50.721591 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:50.730255 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0af5fb7-57f5-48aa-bee7-fd9d6cdbf14d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:50.731701 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0af5fb7-57f5-48aa-bee7-fd9d6cdbf14d None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:50.749678 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-a0af5fb7-57f5-48aa-bee7-fd9d6cdbf14d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:50.749678 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:50.749678 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:50.749678 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:50.749678 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:50.749678 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:50.749678 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:50.749678 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:50.749678 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:50.749678 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:50.749678 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:50.749678 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:50.749678 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:27:50.749678 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:27:50.749678 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:27:50.751043 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:27:50.751043 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:50.751043 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:50.751043 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:50.751043 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:50.751043 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:50.751043 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:50.751043 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:50.751043 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:50.751043 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:50.751043 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:50.751043 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:50.751043 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:50.751043 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:50.751043 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:50.751043 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:50.751043 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:50.752077 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:27:50.752077 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:27:50.752077 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:50.752077 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:50.752077 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 290/568] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:27:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:50.752475 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0af5fb7-57f5-48aa-bee7-fd9d6cdbf14d None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:50.770078 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0a7fbc-f117-4fad-8e67-27a9885f9193 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:50.803992 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9ddea1-c870-4fbd-a621-a484a1ba13b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:50.804776 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ddea1-c870-4fbd-a621-a484a1ba13b0 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:50.805094 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ddea1-c870-4fbd-a621-a484a1ba13b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:50.805536 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ddea1-c870-4fbd-a621-a484a1ba13b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:50.812462 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9ddea1-c870-4fbd-a621-a484a1ba13b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:50.813971 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9ddea1-c870-4fbd-a621-a484a1ba13b0 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:50.827844 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-8f9ddea1-c870-4fbd-a621-a484a1ba13b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:27:50.827844 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:50.827844 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:50.827844 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:50.827844 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:50.827844 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:50.827844 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:50.827844 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:50.827844 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:50.827844 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:50.827844 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:50.827844 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:50.827844 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:27:50.827844 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:27:50.827844 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:27:50.828864 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:27:50.828864 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:50.828864 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:50.828864 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:50.828864 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:50.828864 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:50.828864 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:50.828864 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:50.828864 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:50.828864 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:50.828864 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:50.828864 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:50.828864 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:50.828864 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:50.828864 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:50.828864 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:50.828864 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:50.829739 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:27:50.829739 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:27:50.829739 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:27:50.829739 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:27:50.829739 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:27:50.829739 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:27:50.829739 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:27:50.829739 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:27:50.829739 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 279/569] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:27:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:50.830215 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618dc871-4cc3-45d9-8ab2-e2ec1a93fb4c None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:50.830666 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618dc871-4cc3-45d9-8ab2-e2ec1a93fb4c None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:50.830760 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618dc871-4cc3-45d9-8ab2-e2ec1a93fb4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:50.841065 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:50.847546 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-618dc871-4cc3-45d9-8ab2-e2ec1a93fb4c None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:50.848765 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-618dc871-4cc3-45d9-8ab2-e2ec1a93fb4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:50.860106 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0af5fb7-57f5-48aa-bee7-fd9d6cdbf14d None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:50.894961 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb25eb30-c85f-4d0f-9f25-7db8d2b8f111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:50.895807 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25eb30-c85f-4d0f-9f25-7db8d2b8f111 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:50.896092 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25eb30-c85f-4d0f-9f25-7db8d2b8f111 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:50.896371 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25eb30-c85f-4d0f-9f25-7db8d2b8f111 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:50.907001 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb25eb30-c85f-4d0f-9f25-7db8d2b8f111 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:50.908398 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb25eb30-c85f-4d0f-9f25-7db8d2b8f111 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:50.916108 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 280/570] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:27:50.916942 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618dc871-4cc3-45d9-8ab2-e2ec1a93fb4c None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:50.925790 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-bb25eb30-c85f-4d0f-9f25-7db8d2b8f111 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:50.925790 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:50.925790 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:50.925790 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:50.925790 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:50.925790 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:50.925790 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:50.925790 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:50.925790 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:50.925790 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:50.925790 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:50.925790 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:50.925790 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:27:50.925790 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:27:50.925790 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:27:50.932951 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:27:50.932951 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:50.932951 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:50.932951 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:50.932951 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:50.932951 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:50.932951 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:50.932951 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:50.932951 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:50.932951 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:50.932951 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:50.932951 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:50.932951 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:50.932951 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:50.932951 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:50.932951 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:50.932951 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:50.934384 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:27:50.934384 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:27:50.934384 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:50.934384 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:50.934384 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 291/571] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:27:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:50.934384 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb25eb30-c85f-4d0f-9f25-7db8d2b8f111 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:50.978220 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:51.029904 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb25eb30-c85f-4d0f-9f25-7db8d2b8f111 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:51.035403 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618dc871-4cc3-45d9-8ab2-e2ec1a93fb4c None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:51.064504 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7e6e05-1965-4870-aa98-cb65f3fbacfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:51.065161 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e6e05-1965-4870-aa98-cb65f3fbacfa None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:51.065448 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e6e05-1965-4870-aa98-cb65f3fbacfa None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:51.065751 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e6e05-1965-4870-aa98-cb65f3fbacfa None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:51.066476 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7e6e05-1965-4870-aa98-cb65f3fbacfa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:51.066783 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7e6e05-1965-4870-aa98-cb65f3fbacfa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:51.068161 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7e6e05-1965-4870-aa98-cb65f3fbacfa None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:51.069573 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:51.070186 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a908248-8931-4bed-99c6-4127fe601be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:51.071657 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a908248-8931-4bed-99c6-4127fe601be5 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:51.071955 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a908248-8931-4bed-99c6-4127fe601be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:51.072315 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a908248-8931-4bed-99c6-4127fe601be5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:51.081436 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 292/572] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:27:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:51.082285 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7e6e05-1965-4870-aa98-cb65f3fbacfa None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:51.083852 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a908248-8931-4bed-99c6-4127fe601be5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:27:51.091696 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-5a908248-8931-4bed-99c6-4127fe601be5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:27:51.091696 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:51.091696 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:51.091696 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:51.091696 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:51.091696 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:51.091696 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:51.091696 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:51.091696 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:51.091696 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:51.091696 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:51.091696 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:51.091696 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:27:51.091696 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:27:51.091696 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:27:51.091696 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:27:51.092645 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:27:51.092645 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:27:51.092645 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:27:51.092645 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:27:51.092645 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:51.092645 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:51.092645 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:51.092645 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:51.092645 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:51.092645 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:51.092645 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:51.092645 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:51.092645 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:51.092645 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:51.092645 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:51.092645 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:51.093495 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:51.093495 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:51.093495 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:51.093495 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:51.093495 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:27:51.093495 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:27:51.093495 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:27:51.093495 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:27:51.093495 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:27:51.093495 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:27:51.093495 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:27:51.093495 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:27:51.093495 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 281/573] 10.222.0.13 () {56 vars in 1102 bytes} [Fri Jun 24 08:27:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:51.094223 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a908248-8931-4bed-99c6-4127fe601be5 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:51.187484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7e6e05-1965-4870-aa98-cb65f3fbacfa None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:51.191423 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a908248-8931-4bed-99c6-4127fe601be5 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:51.199438 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:51.221059 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89891448-39c1-4511-a3c3-a8b323986965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:51.221734 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89891448-39c1-4511-a3c3-a8b323986965 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:51.222016 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89891448-39c1-4511-a3c3-a8b323986965 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:51.222286 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89891448-39c1-4511-a3c3-a8b323986965 None admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:51.223056 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89891448-39c1-4511-a3c3-a8b323986965 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:51.223359 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89891448-39c1-4511-a3c3-a8b323986965 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:51.224777 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89891448-39c1-4511-a3c3-a8b323986965 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:51.226045 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16fa70e-7738-4e1f-bc9b-3522cb807abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:51.226702 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16fa70e-7738-4e1f-bc9b-3522cb807abe None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:51.227030 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16fa70e-7738-4e1f-bc9b-3522cb807abe None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:51.227314 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16fa70e-7738-4e1f-bc9b-3522cb807abe None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:51.228028 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16fa70e-7738-4e1f-bc9b-3522cb807abe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:51.228336 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16fa70e-7738-4e1f-bc9b-3522cb807abe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:51.229721 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16fa70e-7738-4e1f-bc9b-3522cb807abe None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:51.236791 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 293/574] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Jun 24 08:27:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:51.237569 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89891448-39c1-4511-a3c3-a8b323986965 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:51.245643 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 282/575] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:27:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:51.246772 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16fa70e-7738-4e1f-bc9b-3522cb807abe None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:51.294149 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:51.340196 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89891448-39c1-4511-a3c3-a8b323986965 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:51.347564 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16fa70e-7738-4e1f-bc9b-3522cb807abe None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:51.376490 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dead384-c197-48ed-8fb5-bcf519a1013d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:51.377288 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dead384-c197-48ed-8fb5-bcf519a1013d None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:51.377578 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dead384-c197-48ed-8fb5-bcf519a1013d None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:51.377881 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dead384-c197-48ed-8fb5-bcf519a1013d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:51.382681 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bab8a2-6fd4-4149-8062-7d17200876c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:51.383695 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bab8a2-6fd4-4149-8062-7d17200876c4 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:51.383979 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bab8a2-6fd4-4149-8062-7d17200876c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:51.384260 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bab8a2-6fd4-4149-8062-7d17200876c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:51.388561 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dead384-c197-48ed-8fb5-bcf519a1013d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:27:51.390898 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bab8a2-6fd4-4149-8062-7d17200876c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:51.392341 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bab8a2-6fd4-4149-8062-7d17200876c4 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:51.396199 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-4dead384-c197-48ed-8fb5-bcf519a1013d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:27:51.396199 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:51.396199 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:51.396199 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:51.396199 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:51.396199 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:51.396199 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:51.396199 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:51.396199 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:51.396199 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:51.396199 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:51.396199 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:51.396199 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:27:51.396199 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:27:51.396199 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:27:51.396199 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:27:51.397123 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:27:51.397123 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:27:51.397123 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:27:51.397123 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:27:51.397123 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:51.397123 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:51.397123 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:51.397123 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:51.397123 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:51.397123 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:51.397123 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:51.397123 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:51.397123 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:51.397123 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:51.397123 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:51.397123 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:51.398012 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:51.398012 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:51.398012 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:51.398012 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:51.398012 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:27:51.398012 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:27:51.398012 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:27:51.398012 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:27:51.398012 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:27:51.398012 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:27:51.398012 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:27:51.398012 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:51.398012 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 294/576] 10.222.0.13 () {56 vars in 1102 bytes} [Fri Jun 24 08:27:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:51.398809 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dead384-c197-48ed-8fb5-bcf519a1013d None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:51.407507 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-19bab8a2-6fd4-4149-8062-7d17200876c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:27:51.407507 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:51.407507 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:51.407507 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:51.407507 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:51.407507 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:51.407507 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:51.407507 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:51.407507 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:51.407507 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:51.407507 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:51.407507 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:51.407507 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:27:51.407507 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:27:51.407507 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:27:51.408444 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:27:51.408444 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:51.408444 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:51.408444 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:51.408444 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:51.408444 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:51.408444 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:51.408444 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:51.408444 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:51.408444 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:51.408444 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:51.408444 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:51.408444 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:51.408444 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:51.408444 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:51.408444 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:51.408444 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:51.409325 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:27:51.409325 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:27:51.409325 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:27:51.409325 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:27:51.409325 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:27:51.409325 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:27:51.409325 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:27:51.409325 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:27:51.409325 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 283/577] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:27:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:51.410194 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bab8a2-6fd4-4149-8062-7d17200876c4 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:51.440044 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:51.505517 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dead384-c197-48ed-8fb5-bcf519a1013d None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:51.513710 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bab8a2-6fd4-4149-8062-7d17200876c4 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:51.519319 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:51.543927 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ecbf2b-4f4c-48af-ad56-9238584367ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:51.544593 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ecbf2b-4f4c-48af-ad56-9238584367ae None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:51.544879 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ecbf2b-4f4c-48af-ad56-9238584367ae None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:51.545152 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ecbf2b-4f4c-48af-ad56-9238584367ae None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:51.545929 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ecbf2b-4f4c-48af-ad56-9238584367ae None admin] RBAC: Adding query filter params (name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:51.546293 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ecbf2b-4f4c-48af-ad56-9238584367ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:51.547697 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ecbf2b-4f4c-48af-ad56-9238584367ae None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:51.553608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f4344-68e9-40f3-b318-1bcb5ec81a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:51.554518 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f4344-68e9-40f3-b318-1bcb5ec81a5c None admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:51.554801 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f4344-68e9-40f3-b318-1bcb5ec81a5c None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:51.555176 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f4344-68e9-40f3-b318-1bcb5ec81a5c None admin] PATH_INFO: `/v3/projects/180b0af5863349e99ae412b07c1e9465/users/74bb1bc6821b4500b26f160ccae5b7d5/roles/6ca1ae4292d9414ca6526c6e8608de9e` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:51.556430 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 295/578] 10.222.0.13 () {56 vars in 1103 bytes} [Fri Jun 24 08:27:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:51.557210 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ecbf2b-4f4c-48af-ad56-9238584367ae None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:51.572929 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f4344-68e9-40f3-b318-1bcb5ec81a5c None admin] RBAC: Authorizing `identity:create_grant(project_id=180b0af5863349e99ae412b07c1e9465, user_id=74bb1bc6821b4500b26f160ccae5b7d5, role_id=6ca1ae4292d9414ca6526c6e8608de9e)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:51.574427 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f4344-68e9-40f3-b318-1bcb5ec81a5c None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:51.587521 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 284/579] 10.222.0.13 () {58 vars in 1439 bytes} [Fri Jun 24 08:27:51 2022] PUT /identity/v3/projects/180b0af5863349e99ae412b07c1e9465/users/74bb1bc6821b4500b26f160ccae5b7d5/roles/6ca1ae4292d9414ca6526c6e8608de9e => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:27:51.588289 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f4344-68e9-40f3-b318-1bcb5ec81a5c None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:51.671128 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ecbf2b-4f4c-48af-ad56-9238584367ae None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:51.689173 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:51.715246 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0461645b-19ff-4eb2-9765-eb510b64feac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:51.716373 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461645b-19ff-4eb2-9765-eb510b64feac None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:51.716662 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461645b-19ff-4eb2-9765-eb510b64feac None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:51.716935 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461645b-19ff-4eb2-9765-eb510b64feac None admin] PATH_INFO: `/v3/users/glance` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:51.721188 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f4344-68e9-40f3-b318-1bcb5ec81a5c None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:51.725311 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0461645b-19ff-4eb2-9765-eb510b64feac None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:51.726808 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0461645b-19ff-4eb2-9765-eb510b64feac None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:51.745709 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-0461645b-19ff-4eb2-9765-eb510b64feac None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 24 08:27:51.745709 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:51.745709 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:51.745709 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:51.745709 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:51.745709 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:51.745709 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:51.745709 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:51.745709 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:51.745709 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:51.745709 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:51.745709 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:51.745709 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:27:51.745709 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:27:51.745709 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:27:51.745709 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:27:51.746677 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:51.746677 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:51.746677 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:27:51.746677 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:27:51.746677 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:27:51.746677 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:27:51.746677 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:51.746677 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:51.746677 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:51.746677 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:51.746677 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:51.746677 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:51.746677 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:51.746677 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:51.746677 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:51.746677 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:51.746677 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:51.746677 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:51.747672 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:51.747672 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:51.747672 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:27:51.747672 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:27:51.747672 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:27:51.747672 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:27:51.747672 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:27:51.747672 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:27:51.747672 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 24 08:27:51.747672 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:51.749006 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 296/580] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:27:51 2022] GET /identity/v3/users/glance => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:51.749195 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0461645b-19ff-4eb2-9765-eb510b64feac None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:51.774593 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43c7821-a46e-43cd-b213-8c7eb67b2440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:51.775802 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43c7821-a46e-43cd-b213-8c7eb67b2440 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:51.776078 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43c7821-a46e-43cd-b213-8c7eb67b2440 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:51.776355 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43c7821-a46e-43cd-b213-8c7eb67b2440 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:51.777111 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43c7821-a46e-43cd-b213-8c7eb67b2440 None admin] RBAC: Adding query filter params (name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:51.777418 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43c7821-a46e-43cd-b213-8c7eb67b2440 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:51.779812 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43c7821-a46e-43cd-b213-8c7eb67b2440 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:51.793121 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 285/581] 10.222.0.13 () {56 vars in 1103 bytes} [Fri Jun 24 08:27:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:51.793667 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:51.794475 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43c7821-a46e-43cd-b213-8c7eb67b2440 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:51.872393 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0461645b-19ff-4eb2-9765-eb510b64feac None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:51.904843 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43c7821-a46e-43cd-b213-8c7eb67b2440 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:51.910157 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5ea3e1-4840-4a8a-8de0-3db28c550cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:51.910822 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ea3e1-4840-4a8a-8de0-3db28c550cc8 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:51.911105 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ea3e1-4840-4a8a-8de0-3db28c550cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:51.911378 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ea3e1-4840-4a8a-8de0-3db28c550cc8 None admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:51.912170 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5ea3e1-4840-4a8a-8de0-3db28c550cc8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:51.912483 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5ea3e1-4840-4a8a-8de0-3db28c550cc8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:51.913921 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5ea3e1-4840-4a8a-8de0-3db28c550cc8 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:51.927484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 297/582] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Jun 24 08:27:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:51.928312 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5ea3e1-4840-4a8a-8de0-3db28c550cc8 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:51.951852 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3fe938-10e3-48ef-987e-4d57160df22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:51.952661 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3fe938-10e3-48ef-987e-4d57160df22a None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:51.952976 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3fe938-10e3-48ef-987e-4d57160df22a None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:51.953255 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3fe938-10e3-48ef-987e-4d57160df22a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:51.960028 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:51.966123 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3fe938-10e3-48ef-987e-4d57160df22a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:51.967522 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3fe938-10e3-48ef-987e-4d57160df22a None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:51.984970 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-1f3fe938-10e3-48ef-987e-4d57160df22a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:51.984970 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:51.984970 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:51.984970 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:51.984970 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:51.984970 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:51.984970 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:51.984970 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:51.984970 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:51.984970 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:51.984970 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:51.984970 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:51.984970 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:27:51.984970 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:27:51.984970 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:27:51.985974 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:27:51.985974 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:51.985974 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:51.985974 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:51.985974 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:51.985974 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:51.985974 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:51.985974 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:51.985974 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:51.985974 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:51.985974 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:51.985974 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:51.985974 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:51.985974 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:51.985974 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:51.985974 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:51.985974 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:51.986848 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:27:51.986848 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:27:51.986848 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:51.986848 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:27:51.986848 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 286/583] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:27:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:51.987126 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3fe938-10e3-48ef-987e-4d57160df22a None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:52.047710 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:52.054789 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5ea3e1-4840-4a8a-8de0-3db28c550cc8 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:52.094536 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a620c-4a81-4991-9af6-184a8d0b1953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:52.095211 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a620c-4a81-4991-9af6-184a8d0b1953 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:52.095495 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a620c-4a81-4991-9af6-184a8d0b1953 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:52.095859 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a620c-4a81-4991-9af6-184a8d0b1953 None admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:52.096607 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55a620c-4a81-4991-9af6-184a8d0b1953 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:52.100875 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55a620c-4a81-4991-9af6-184a8d0b1953 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:52.103078 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55a620c-4a81-4991-9af6-184a8d0b1953 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:52.113442 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3fe938-10e3-48ef-987e-4d57160df22a None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:52.135043 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 298/584] 10.222.0.13 () {56 vars in 1137 bytes} [Fri Jun 24 08:27:51 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:52.135043 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a620c-4a81-4991-9af6-184a8d0b1953 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:52.165779 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2970b19e-a349-401b-866b-61304e04555a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:52.166597 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2970b19e-a349-401b-866b-61304e04555a None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:52.166916 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2970b19e-a349-401b-866b-61304e04555a None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:52.167223 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2970b19e-a349-401b-866b-61304e04555a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:52.178844 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2970b19e-a349-401b-866b-61304e04555a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:52.181102 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2970b19e-a349-401b-866b-61304e04555a None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:52.199593 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-2970b19e-a349-401b-866b-61304e04555a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:52.199593 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:52.199593 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:52.199593 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:52.199593 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:52.199593 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:52.199593 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:52.199593 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:52.199593 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:52.199593 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:52.199593 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:52.199593 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:52.199593 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:27:52.199593 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:27:52.199593 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:27:52.201015 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:27:52.201015 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:52.201015 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:52.201015 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:52.201015 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:52.201015 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:52.201015 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:52.201015 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:52.201015 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:52.201015 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:52.201015 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:52.201015 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:52.201015 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:52.201015 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:52.201015 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:52.201015 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:52.201015 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:52.201935 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:27:52.201935 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:27:52.201935 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:52.201935 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:27:52.201935 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 287/585] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:27:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:52.201935 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2970b19e-a349-401b-866b-61304e04555a None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:52.248011 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a620c-4a81-4991-9af6-184a8d0b1953 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:52.251645 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:52.286954 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc167b47-6e86-418b-a3cf-39656483b8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:52.287609 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc167b47-6e86-418b-a3cf-39656483b8f4 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:52.287914 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc167b47-6e86-418b-a3cf-39656483b8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:52.288552 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc167b47-6e86-418b-a3cf-39656483b8f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:52.290008 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc167b47-6e86-418b-a3cf-39656483b8f4 None admin] RBAC: Adding query filter params (role.id=6ca1ae4292d9414ca6526c6e8608de9e, scope.project.id=180b0af5863349e99ae412b07c1e9465, user.id=bec8c2f113cf41a0939b81f38fbee813) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:52.290334 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc167b47-6e86-418b-a3cf-39656483b8f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:52.291782 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc167b47-6e86-418b-a3cf-39656483b8f4 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:52.297611 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 299/586] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Jun 24 08:27:52 2022] GET /identity/v3/role_assignments?user.id=bec8c2f113cf41a0939b81f38fbee813&scope.project.id=180b0af5863349e99ae412b07c1e9465&role.id=6ca1ae4292d9414ca6526c6e8608de9e => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:52.298416 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc167b47-6e86-418b-a3cf-39656483b8f4 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:52.329054 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:52.333674 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2970b19e-a349-401b-866b-61304e04555a None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:52.418532 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736d588c-32a8-4e7c-a579-3e93b09caed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:52.419935 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736d588c-32a8-4e7c-a579-3e93b09caed7 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:52.420886 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736d588c-32a8-4e7c-a579-3e93b09caed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:52.421361 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736d588c-32a8-4e7c-a579-3e93b09caed7 None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:52.422628 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736d588c-32a8-4e7c-a579-3e93b09caed7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:52.423122 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736d588c-32a8-4e7c-a579-3e93b09caed7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:52.425499 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736d588c-32a8-4e7c-a579-3e93b09caed7 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:52.455502 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 288/587] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:27:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:52.456179 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736d588c-32a8-4e7c-a579-3e93b09caed7 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:52.505648 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc167b47-6e86-418b-a3cf-39656483b8f4 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:52.545768 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49881bd-f951-4739-9ead-f192066271a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:52.548290 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49881bd-f951-4739-9ead-f192066271a2 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:52.548804 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49881bd-f951-4739-9ead-f192066271a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:52.549280 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49881bd-f951-4739-9ead-f192066271a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:52.567452 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49881bd-f951-4739-9ead-f192066271a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:52.569229 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49881bd-f951-4739-9ead-f192066271a2 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:52.600804 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-e49881bd-f951-4739-9ead-f192066271a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:52.600804 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:52.600804 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:52.600804 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:52.600804 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:52.600804 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:52.600804 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:52.600804 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:52.600804 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:52.600804 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:52.600804 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:52.600804 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:52.600804 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:27:52.600804 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:27:52.600804 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:27:52.601763 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:27:52.601763 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:52.601763 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:52.601763 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:52.601763 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:52.601763 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:52.601763 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:52.601763 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:52.601763 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:52.601763 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:52.601763 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:52.601763 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:52.601763 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:52.601763 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:52.601763 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:52.601763 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:52.601763 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:52.603395 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:27:52.603395 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:27:52.603395 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:52.603395 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:52.609785 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 300/588] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:27:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:52.610829 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49881bd-f951-4739-9ead-f192066271a2 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:52.617190 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736d588c-32a8-4e7c-a579-3e93b09caed7 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:52.619471 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:52.656558 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49ffca5-b1fe-48d7-a12a-faef1a1556a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:52.657586 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ffca5-b1fe-48d7-a12a-faef1a1556a1 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:52.658251 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ffca5-b1fe-48d7-a12a-faef1a1556a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:52.658732 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ffca5-b1fe-48d7-a12a-faef1a1556a1 None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:52.659731 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49ffca5-b1fe-48d7-a12a-faef1a1556a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:52.660239 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49ffca5-b1fe-48d7-a12a-faef1a1556a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:52.661859 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49ffca5-b1fe-48d7-a12a-faef1a1556a1 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:52.674802 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 289/589] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:27:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:52.676120 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49ffca5-b1fe-48d7-a12a-faef1a1556a1 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:52.778650 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:52.791414 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49ffca5-b1fe-48d7-a12a-faef1a1556a1 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:52.836704 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fda795-4704-4a3b-b70b-3408b8ae02ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:52.837785 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fda795-4704-4a3b-b70b-3408b8ae02ca None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:52.838295 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fda795-4704-4a3b-b70b-3408b8ae02ca None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:52.838875 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fda795-4704-4a3b-b70b-3408b8ae02ca None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:52.839798 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fda795-4704-4a3b-b70b-3408b8ae02ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:52.840304 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fda795-4704-4a3b-b70b-3408b8ae02ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:52.843010 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49881bd-f951-4739-9ead-f192066271a2 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:52.843599 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fda795-4704-4a3b-b70b-3408b8ae02ca None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:52.859359 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 290/590] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:27:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:52.865118 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fda795-4704-4a3b-b70b-3408b8ae02ca None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:52.890841 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e201d282-efef-4c19-a56a-308de52bf3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:52.891654 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e201d282-efef-4c19-a56a-308de52bf3d0 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:52.891938 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e201d282-efef-4c19-a56a-308de52bf3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:52.892210 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e201d282-efef-4c19-a56a-308de52bf3d0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:52.906814 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e201d282-efef-4c19-a56a-308de52bf3d0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:52.909147 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e201d282-efef-4c19-a56a-308de52bf3d0 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:52.927647 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:52.938769 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-e201d282-efef-4c19-a56a-308de52bf3d0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 08:27:52.938769 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:52.938769 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:52.938769 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:52.938769 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:52.938769 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:52.938769 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:52.938769 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:52.938769 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:52.938769 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:52.938769 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:52.938769 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:52.938769 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:27:52.938769 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:27:52.938769 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:27:52.938769 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:27:52.939735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:52.939735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:52.939735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:27:52.939735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:27:52.939735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:27:52.939735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:27:52.939735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:52.939735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:52.939735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:52.939735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:52.939735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:52.939735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:52.939735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:52.939735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:52.939735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:52.939735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:52.939735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:52.939735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:52.940664 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:52.940664 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:52.940664 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:27:52.940664 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:27:52.940664 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:27:52.940664 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:27:52.940664 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:27:52.940664 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:27:52.940664 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 08:27:52.940664 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:52.940664 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 301/591] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:27:52 2022] GET /identity/v3/users/cinder => generated 84 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:52.941247 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e201d282-efef-4c19-a56a-308de52bf3d0 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:53.046501 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fda795-4704-4a3b-b70b-3408b8ae02ca None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:53.092249 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e201d282-efef-4c19-a56a-308de52bf3d0 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:53.092637 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be7452-c7c6-4849-ae7e-6569afd16f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:53.093441 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be7452-c7c6-4849-ae7e-6569afd16f5f None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:53.093748 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be7452-c7c6-4849-ae7e-6569afd16f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:53.094030 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be7452-c7c6-4849-ae7e-6569afd16f5f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:53.104604 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5be7452-c7c6-4849-ae7e-6569afd16f5f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:53.106558 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5be7452-c7c6-4849-ae7e-6569afd16f5f None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:53.129365 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:53.129915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-b5be7452-c7c6-4849-ae7e-6569afd16f5f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 08:27:53.129915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:53.129915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:53.129915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:53.129915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:53.129915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:53.129915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:53.129915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:53.129915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:53.129915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:53.129915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:53.129915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:53.129915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:27:53.129915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:27:53.129915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:27:53.129915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:27:53.130802 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:53.130802 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:53.130802 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:27:53.130802 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:27:53.130802 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:27:53.130802 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:27:53.130802 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:53.130802 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:53.130802 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:53.130802 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:53.130802 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:53.130802 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:53.130802 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:53.130802 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:53.130802 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:53.130802 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:53.130802 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:53.130802 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:53.131729 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:53.131729 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:53.131729 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:27:53.131729 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:27:53.131729 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:27:53.131729 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:27:53.131729 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:27:53.131729 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:27:53.131729 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 08:27:53.131729 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:27:53.131729 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 291/592] 10.222.0.13 () {56 vars in 1102 bytes} [Fri Jun 24 08:27:52 2022] GET /identity/v3/users/neutron => generated 85 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:53.132298 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be7452-c7c6-4849-ae7e-6569afd16f5f None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:53.142218 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bcf936-433a-43dd-9e9d-de691d5f8001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:53.143517 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bcf936-433a-43dd-9e9d-de691d5f8001 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:53.144539 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bcf936-433a-43dd-9e9d-de691d5f8001 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:53.145026 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bcf936-433a-43dd-9e9d-de691d5f8001 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:53.154764 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bcf936-433a-43dd-9e9d-de691d5f8001 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:53.156452 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bcf936-433a-43dd-9e9d-de691d5f8001 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:53.171847 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-20bcf936-433a-43dd-9e9d-de691d5f8001 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:27:53.171847 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:53.171847 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:53.171847 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:53.171847 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:53.171847 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:53.171847 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:53.171847 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:53.171847 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:53.171847 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:53.171847 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:53.171847 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:53.171847 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:27:53.171847 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:27:53.171847 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:27:53.172734 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:27:53.172734 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:53.172734 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:53.172734 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:53.172734 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:53.172734 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:53.172734 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:53.172734 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:53.172734 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:53.172734 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:53.172734 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:53.172734 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:53.172734 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:53.172734 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:53.172734 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:53.172734 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:53.172734 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:53.173624 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:27:53.173624 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:27:53.173624 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:27:53.173624 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:27:53.173624 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:27:53.173624 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:27:53.173624 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:27:53.173624 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:53.177719 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 302/593] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:27:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:53.178550 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bcf936-433a-43dd-9e9d-de691d5f8001 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:53.231540 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:53.252769 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be7452-c7c6-4849-ae7e-6569afd16f5f None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:53.289018 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd89950a-9c6a-4fe8-9c2e-f6450a10d82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:53.290054 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bcf936-433a-43dd-9e9d-de691d5f8001 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:53.290983 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd89950a-9c6a-4fe8-9c2e-f6450a10d82d None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:53.291471 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd89950a-9c6a-4fe8-9c2e-f6450a10d82d None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:53.291947 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd89950a-9c6a-4fe8-9c2e-f6450a10d82d None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:53.293009 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd89950a-9c6a-4fe8-9c2e-f6450a10d82d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:53.293522 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd89950a-9c6a-4fe8-9c2e-f6450a10d82d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:53.295384 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd89950a-9c6a-4fe8-9c2e-f6450a10d82d None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:53.314403 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 292/594] 10.222.0.13 () {56 vars in 1137 bytes} [Fri Jun 24 08:27:53 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:53.315559 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd89950a-9c6a-4fe8-9c2e-f6450a10d82d None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:53.327719 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3936af86-7569-4b16-9e76-ab9a1017c0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:53.328404 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3936af86-7569-4b16-9e76-ab9a1017c0ac None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:53.328688 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3936af86-7569-4b16-9e76-ab9a1017c0ac None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:53.328965 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3936af86-7569-4b16-9e76-ab9a1017c0ac None admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:53.329924 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3936af86-7569-4b16-9e76-ab9a1017c0ac None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:53.330153 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3936af86-7569-4b16-9e76-ab9a1017c0ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:53.331595 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3936af86-7569-4b16-9e76-ab9a1017c0ac None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:53.351468 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 303/595] 10.222.0.13 () {56 vars in 1139 bytes} [Fri Jun 24 08:27:53 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:53.358178 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3936af86-7569-4b16-9e76-ab9a1017c0ac None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:53.418650 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:53.429513 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd89950a-9c6a-4fe8-9c2e-f6450a10d82d None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:53.472685 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39231bfd-d989-454d-ba8f-310b2fcbe5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:53.473842 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39231bfd-d989-454d-ba8f-310b2fcbe5a9 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:53.474337 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39231bfd-d989-454d-ba8f-310b2fcbe5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:53.474862 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39231bfd-d989-454d-ba8f-310b2fcbe5a9 None admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:53.475918 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39231bfd-d989-454d-ba8f-310b2fcbe5a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:53.476499 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39231bfd-d989-454d-ba8f-310b2fcbe5a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:53.478146 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3936af86-7569-4b16-9e76-ab9a1017c0ac None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:53.479032 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39231bfd-d989-454d-ba8f-310b2fcbe5a9 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:53.488859 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:53.493609 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 293/596] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Jun 24 08:27:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:53.494738 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39231bfd-d989-454d-ba8f-310b2fcbe5a9 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:53.541970 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a282255-30d2-4d02-816d-f18be1dd0fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:53.542876 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a282255-30d2-4d02-816d-f18be1dd0fad None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:53.543207 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a282255-30d2-4d02-816d-f18be1dd0fad None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:53.543490 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a282255-30d2-4d02-816d-f18be1dd0fad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:53.555179 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a282255-30d2-4d02-816d-f18be1dd0fad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:53.557214 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a282255-30d2-4d02-816d-f18be1dd0fad None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:53.578101 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-6a282255-30d2-4d02-816d-f18be1dd0fad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:53.578101 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:53.578101 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:53.578101 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:53.578101 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:53.578101 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:53.578101 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:53.578101 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:53.578101 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:53.578101 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:53.578101 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:53.578101 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:53.578101 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:27:53.578101 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:27:53.578101 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:27:53.579039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:27:53.579039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:53.579039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:53.579039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:53.579039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:53.579039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:53.579039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:53.579039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:53.579039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:53.579039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:53.579039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:53.579039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:53.579039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:53.579039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:53.579039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:53.579039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:53.579039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:53.579945 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:27:53.579945 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:27:53.579945 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:53.579945 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:53.579945 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 304/597] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:27:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:53.580662 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a282255-30d2-4d02-816d-f18be1dd0fad None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:53.619831 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39231bfd-d989-454d-ba8f-310b2fcbe5a9 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:53.659693 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c2a8c-3420-4e75-9697-c83d28107ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:53.661026 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c2a8c-3420-4e75-9697-c83d28107ef2 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:53.661527 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c2a8c-3420-4e75-9697-c83d28107ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:53.662028 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c2a8c-3420-4e75-9697-c83d28107ef2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:53.673850 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90c2a8c-3420-4e75-9697-c83d28107ef2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:53.675624 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90c2a8c-3420-4e75-9697-c83d28107ef2 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:53.690485 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:53.693668 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a282255-30d2-4d02-816d-f18be1dd0fad None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:53.696384 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-a90c2a8c-3420-4e75-9697-c83d28107ef2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:53.696384 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:53.696384 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:53.696384 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:53.696384 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:53.696384 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:53.696384 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:53.696384 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:53.696384 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:53.696384 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:53.696384 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:53.696384 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:53.696384 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:27:53.696384 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:27:53.696384 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:27:53.697291 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:27:53.697291 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:53.697291 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:53.697291 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:53.697291 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:53.697291 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:53.697291 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:53.697291 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:53.697291 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:53.697291 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:53.697291 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:53.697291 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:53.697291 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:53.697291 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:53.697291 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:53.697291 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:53.697291 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:53.698207 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:27:53.698207 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:27:53.698207 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:53.698207 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:27:53.700425 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 294/598] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:27:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:53.701439 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c2a8c-3420-4e75-9697-c83d28107ef2 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:53.736395 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3289a7-c7df-4f47-9532-8effbe64760b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:53.737290 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3289a7-c7df-4f47-9532-8effbe64760b None admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:53.737684 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3289a7-c7df-4f47-9532-8effbe64760b None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:53.737986 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3289a7-c7df-4f47-9532-8effbe64760b None admin] PATH_INFO: `/v3/projects/180b0af5863349e99ae412b07c1e9465/users/d15c5f6aeb874191b73c8ee0410c6bc1/roles/6ca1ae4292d9414ca6526c6e8608de9e` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:53.754713 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3289a7-c7df-4f47-9532-8effbe64760b None admin] RBAC: Authorizing `identity:create_grant(project_id=180b0af5863349e99ae412b07c1e9465, user_id=d15c5f6aeb874191b73c8ee0410c6bc1, role_id=6ca1ae4292d9414ca6526c6e8608de9e)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:53.756149 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3289a7-c7df-4f47-9532-8effbe64760b None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:53.760511 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:53.781791 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 305/599] 10.222.0.13 () {58 vars in 1439 bytes} [Fri Jun 24 08:27:53 2022] PUT /identity/v3/projects/180b0af5863349e99ae412b07c1e9465/users/d15c5f6aeb874191b73c8ee0410c6bc1/roles/6ca1ae4292d9414ca6526c6e8608de9e => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:27:53.782839 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3289a7-c7df-4f47-9532-8effbe64760b None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:53.827677 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c2a8c-3420-4e75-9697-c83d28107ef2 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:53.885908 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337e1e7e-f214-4dae-ab91-6e559315f908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:53.887429 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e1e7e-f214-4dae-ab91-6e559315f908 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:53.888169 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e1e7e-f214-4dae-ab91-6e559315f908 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:53.888633 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e1e7e-f214-4dae-ab91-6e559315f908 None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:53.889596 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337e1e7e-f214-4dae-ab91-6e559315f908 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:53.890353 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337e1e7e-f214-4dae-ab91-6e559315f908 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:53.892717 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337e1e7e-f214-4dae-ab91-6e559315f908 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:53.916930 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 295/600] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:27:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:53.916930 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853cb0c4-9d11-4259-947f-11186bf0ebb1 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:53.916930 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853cb0c4-9d11-4259-947f-11186bf0ebb1 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:53.916930 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853cb0c4-9d11-4259-947f-11186bf0ebb1 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:53.920044 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3289a7-c7df-4f47-9532-8effbe64760b None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:53.921055 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 296/601] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:53.922463 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853cb0c4-9d11-4259-947f-11186bf0ebb1 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:53.970986 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44218a65-8cb7-45f8-a14d-dd83b5644011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:53.971659 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44218a65-8cb7-45f8-a14d-dd83b5644011 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:53.972020 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44218a65-8cb7-45f8-a14d-dd83b5644011 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:53.972225 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44218a65-8cb7-45f8-a14d-dd83b5644011 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:53.972954 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44218a65-8cb7-45f8-a14d-dd83b5644011 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:53.973418 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44218a65-8cb7-45f8-a14d-dd83b5644011 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:53.975657 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44218a65-8cb7-45f8-a14d-dd83b5644011 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:53.992438 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 306/602] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:27:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:53.994377 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166da361-f89d-4a9d-9c43-97dd11c6e92b None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:53.994670 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166da361-f89d-4a9d-9c43-97dd11c6e92b None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:53.994951 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166da361-f89d-4a9d-9c43-97dd11c6e92b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:54.014493 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-166da361-f89d-4a9d-9c43-97dd11c6e92b None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:54.016066 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-166da361-f89d-4a9d-9c43-97dd11c6e92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:54.019224 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:54.055976 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853cb0c4-9d11-4259-947f-11186bf0ebb1 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:54.070723 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:54.094546 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369ea84c-2bea-4305-94e2-107393cd4ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:54.095897 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ea84c-2bea-4305-94e2-107393cd4ff3 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:54.096443 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ea84c-2bea-4305-94e2-107393cd4ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:54.096923 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ea84c-2bea-4305-94e2-107393cd4ff3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:54.097752 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 307/603] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:27:54.101762 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166da361-f89d-4a9d-9c43-97dd11c6e92b None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:54.119022 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369ea84c-2bea-4305-94e2-107393cd4ff3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:54.120625 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369ea84c-2bea-4305-94e2-107393cd4ff3 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:54.144155 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-369ea84c-2bea-4305-94e2-107393cd4ff3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:27:54.144155 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:54.144155 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:54.144155 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:54.144155 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:54.144155 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:54.144155 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:54.144155 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:54.144155 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:54.144155 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:54.144155 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:54.144155 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:54.144155 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:27:54.144155 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:27:54.144155 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:27:54.145059 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:27:54.145059 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:54.145059 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:54.145059 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:54.145059 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:54.145059 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:54.145059 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:54.145059 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:54.145059 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:54.145059 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:54.145059 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:54.145059 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:54.145059 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:54.145059 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:54.145059 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:54.145059 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:54.145059 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:54.145958 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:27:54.145958 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:27:54.145958 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:27:54.145958 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:27:54.145958 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:27:54.145958 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:27:54.145958 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:27:54.145958 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:27:54.145958 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 297/604] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:27:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:54.148150 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369ea84c-2bea-4305-94e2-107393cd4ff3 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:54.213257 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166da361-f89d-4a9d-9c43-97dd11c6e92b None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:54.250505 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58796a87-c036-4671-96a9-7a5b5e15eb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:54.251657 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58796a87-c036-4671-96a9-7a5b5e15eb1b None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:54.251941 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58796a87-c036-4671-96a9-7a5b5e15eb1b None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:54.252215 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58796a87-c036-4671-96a9-7a5b5e15eb1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:54.256838 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369ea84c-2bea-4305-94e2-107393cd4ff3 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:54.259993 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58796a87-c036-4671-96a9-7a5b5e15eb1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:54.261445 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58796a87-c036-4671-96a9-7a5b5e15eb1b None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:54.276886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-58796a87-c036-4671-96a9-7a5b5e15eb1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:27:54.276886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:54.276886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:54.276886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:54.276886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:54.276886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:54.276886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:54.276886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:54.276886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:54.276886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:54.276886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:54.276886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:54.276886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:27:54.276886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:27:54.276886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:27:54.278890 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:27:54.278890 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:54.278890 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:54.278890 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:54.278890 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:54.278890 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:54.278890 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:54.278890 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:54.278890 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:54.278890 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:54.278890 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:54.278890 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:54.278890 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:54.278890 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:54.278890 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:54.278890 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:54.278890 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:54.280886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:27:54.280886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:27:54.280886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:27:54.280886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:27:54.280886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:27:54.280886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:27:54.280886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:27:54.280886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:54.285719 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:54.285719 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 308/605] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:27:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:54.285719 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58796a87-c036-4671-96a9-7a5b5e15eb1b None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:54.295133 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb4698-9a8b-4814-9fda-0ae712595009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:54.295939 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb4698-9a8b-4814-9fda-0ae712595009 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:54.296225 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb4698-9a8b-4814-9fda-0ae712595009 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:54.296500 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb4698-9a8b-4814-9fda-0ae712595009 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:54.307520 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eb4698-9a8b-4814-9fda-0ae712595009 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:27:54.315539 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-15eb4698-9a8b-4814-9fda-0ae712595009 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:27:54.315539 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:54.315539 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:54.315539 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:54.315539 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:54.315539 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:54.315539 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:54.315539 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:54.315539 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:54.315539 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:54.315539 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:54.315539 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:54.315539 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:27:54.315539 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:27:54.315539 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:27:54.315539 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:27:54.317589 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:27:54.317589 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:27:54.317589 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:27:54.317589 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:27:54.317589 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:54.317589 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:54.317589 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:54.317589 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:54.317589 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:54.317589 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:54.317589 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:54.317589 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:54.317589 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:54.317589 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:54.317589 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:54.317589 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:54.319536 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:54.319536 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:54.319536 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:54.319536 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:54.319536 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:27:54.319536 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:27:54.319536 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:27:54.319536 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:27:54.319536 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:27:54.319536 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:27:54.319536 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:27:54.319536 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:27:54.327043 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 298/606] 10.222.0.13 () {56 vars in 1102 bytes} [Fri Jun 24 08:27:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:27:54.327043 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb4698-9a8b-4814-9fda-0ae712595009 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:54.398216 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:54.414070 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58796a87-c036-4671-96a9-7a5b5e15eb1b None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:54.434468 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb4698-9a8b-4814-9fda-0ae712595009 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:54.452195 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedefe83-478f-4c3e-8b60-59e331fe0162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:54.452854 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedefe83-478f-4c3e-8b60-59e331fe0162 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:54.453136 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedefe83-478f-4c3e-8b60-59e331fe0162 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:54.453404 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedefe83-478f-4c3e-8b60-59e331fe0162 None admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:54.454222 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedefe83-478f-4c3e-8b60-59e331fe0162 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:54.454531 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedefe83-478f-4c3e-8b60-59e331fe0162 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:54.455959 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedefe83-478f-4c3e-8b60-59e331fe0162 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:54.469571 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 309/607] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Jun 24 08:27:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:54.470481 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedefe83-478f-4c3e-8b60-59e331fe0162 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:54.471909 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606210c9-1524-40b8-b540-0da8bac340de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:54.472579 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606210c9-1524-40b8-b540-0da8bac340de None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:54.472866 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606210c9-1524-40b8-b540-0da8bac340de None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:54.473145 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606210c9-1524-40b8-b540-0da8bac340de None admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:54.474373 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606210c9-1524-40b8-b540-0da8bac340de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:54.474807 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606210c9-1524-40b8-b540-0da8bac340de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:54.476274 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606210c9-1524-40b8-b540-0da8bac340de None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:54.490789 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 299/608] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Jun 24 08:27:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:54.491715 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606210c9-1524-40b8-b540-0da8bac340de None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:54.542887 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:54.589258 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedefe83-478f-4c3e-8b60-59e331fe0162 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:54.594942 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:54.607070 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606210c9-1524-40b8-b540-0da8bac340de None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:54.628780 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7f8bc7-b0e6-487c-9969-fd02586a8401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:54.629485 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f8bc7-b0e6-487c-9969-fd02586a8401 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:54.629795 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f8bc7-b0e6-487c-9969-fd02586a8401 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:54.630071 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f8bc7-b0e6-487c-9969-fd02586a8401 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:54.630818 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7f8bc7-b0e6-487c-9969-fd02586a8401 None admin] RBAC: Adding query filter params (name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:54.631122 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7f8bc7-b0e6-487c-9969-fd02586a8401 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:54.632483 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7f8bc7-b0e6-487c-9969-fd02586a8401 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:54.646058 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 310/609] 10.222.0.13 () {56 vars in 1103 bytes} [Fri Jun 24 08:27:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:54.646706 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b92e8e-54f7-40a6-87b2-a7b15fd1b279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:54.646849 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7f8bc7-b0e6-487c-9969-fd02586a8401 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:54.647737 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b92e8e-54f7-40a6-87b2-a7b15fd1b279 None admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:54.648086 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b92e8e-54f7-40a6-87b2-a7b15fd1b279 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:54.648480 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b92e8e-54f7-40a6-87b2-a7b15fd1b279 None admin] PATH_INFO: `/v3/projects/180b0af5863349e99ae412b07c1e9465/users/44dcdfb5ca1541f9b4967a1f8ddbb34b/roles/6ca1ae4292d9414ca6526c6e8608de9e` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:54.666066 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b92e8e-54f7-40a6-87b2-a7b15fd1b279 None admin] RBAC: Authorizing `identity:create_grant(project_id=180b0af5863349e99ae412b07c1e9465, user_id=44dcdfb5ca1541f9b4967a1f8ddbb34b, role_id=6ca1ae4292d9414ca6526c6e8608de9e)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:54.667591 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b92e8e-54f7-40a6-87b2-a7b15fd1b279 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:54.689807 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 300/610] 10.222.0.13 () {58 vars in 1439 bytes} [Fri Jun 24 08:27:54 2022] PUT /identity/v3/projects/180b0af5863349e99ae412b07c1e9465/users/44dcdfb5ca1541f9b4967a1f8ddbb34b/roles/6ca1ae4292d9414ca6526c6e8608de9e => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:27:54.692558 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a4a65-32d3-429c-95bc-abf3dc23c0e2 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:54.692961 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a4a65-32d3-429c-95bc-abf3dc23c0e2 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:54.693311 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a4a65-32d3-429c-95bc-abf3dc23c0e2 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:54.699766 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 301/611] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:54.699766 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a4a65-32d3-429c-95bc-abf3dc23c0e2 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:54.777969 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7f8bc7-b0e6-487c-9969-fd02586a8401 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:54.807069 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:54.822868 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646659da-0a22-48a5-b2a3-3d8d7fd16443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:54.824185 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646659da-0a22-48a5-b2a3-3d8d7fd16443 None admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:54.824705 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646659da-0a22-48a5-b2a3-3d8d7fd16443 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:54.825184 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646659da-0a22-48a5-b2a3-3d8d7fd16443 None admin] PATH_INFO: `/v3/projects/180b0af5863349e99ae412b07c1e9465/users/c5f7ba6c7faf43e7acbb43eee58d8c8d/roles/6ca1ae4292d9414ca6526c6e8608de9e` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:54.848082 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646659da-0a22-48a5-b2a3-3d8d7fd16443 None admin] RBAC: Authorizing `identity:create_grant(project_id=180b0af5863349e99ae412b07c1e9465, user_id=c5f7ba6c7faf43e7acbb43eee58d8c8d, role_id=6ca1ae4292d9414ca6526c6e8608de9e)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:54.849927 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646659da-0a22-48a5-b2a3-3d8d7fd16443 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:54.861562 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a4a65-32d3-429c-95bc-abf3dc23c0e2 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:54.871634 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 311/612] 10.222.0.13 () {58 vars in 1439 bytes} [Fri Jun 24 08:27:54 2022] PUT /identity/v3/projects/180b0af5863349e99ae412b07c1e9465/users/c5f7ba6c7faf43e7acbb43eee58d8c8d/roles/6ca1ae4292d9414ca6526c6e8608de9e => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:27:54.874109 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b69e9f4-3a3b-4e52-96e2-e33023e32332 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:54.874915 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b69e9f4-3a3b-4e52-96e2-e33023e32332 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:54.875444 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b69e9f4-3a3b-4e52-96e2-e33023e32332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:54.904791 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-8b69e9f4-3a3b-4e52-96e2-e33023e32332 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:54.906636 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-8b69e9f4-3a3b-4e52-96e2-e33023e32332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:54.907257 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5286dcf2-260e-4500-b15c-f80c7cd98ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:54.908066 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5286dcf2-260e-4500-b15c-f80c7cd98ca9 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:54.908352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5286dcf2-260e-4500-b15c-f80c7cd98ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:54.908629 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5286dcf2-260e-4500-b15c-f80c7cd98ca9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:54.927503 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5286dcf2-260e-4500-b15c-f80c7cd98ca9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:54.929359 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5286dcf2-260e-4500-b15c-f80c7cd98ca9 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:54.931628 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:54.956516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-5286dcf2-260e-4500-b15c-f80c7cd98ca9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:54.956516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:54.956516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:54.956516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:54.956516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:54.956516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:54.956516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:54.956516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:54.956516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:54.956516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:54.956516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:54.956516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:54.956516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:27:54.956516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:27:54.956516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:27:54.957648 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:27:54.957648 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:54.957648 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:54.957648 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:54.957648 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:54.957648 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:54.957648 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:54.957648 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:54.957648 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:54.957648 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:54.957648 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:54.957648 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:54.957648 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:54.957648 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:54.957648 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:54.957648 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:54.957648 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:54.958781 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:27:54.958781 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:27:54.958781 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:54.958781 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:27:54.962186 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 302/613] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:27:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:54.964637 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5286dcf2-260e-4500-b15c-f80c7cd98ca9 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:55.050852 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 312/614] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:27:55.082079 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b69e9f4-3a3b-4e52-96e2-e33023e32332 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:55.121198 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5286dcf2-260e-4500-b15c-f80c7cd98ca9 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:55.165338 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf218c4-6502-4fd5-8513-de4d7e088d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:55.166141 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf218c4-6502-4fd5-8513-de4d7e088d8c None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:55.166428 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf218c4-6502-4fd5-8513-de4d7e088d8c None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:55.166701 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf218c4-6502-4fd5-8513-de4d7e088d8c None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:55.167421 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf218c4-6502-4fd5-8513-de4d7e088d8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:55.167724 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf218c4-6502-4fd5-8513-de4d7e088d8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:55.169095 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf218c4-6502-4fd5-8513-de4d7e088d8c None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:55.195988 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 303/615] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:27:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:27:55.201627 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf218c4-6502-4fd5-8513-de4d7e088d8c None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:55.317735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b69e9f4-3a3b-4e52-96e2-e33023e32332 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:55.319308 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:55.328667 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:55.362479 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01490d0f-c36e-4d83-911d-8083812c3c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:55.363748 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01490d0f-c36e-4d83-911d-8083812c3c9f None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:55.364230 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01490d0f-c36e-4d83-911d-8083812c3c9f None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:55.364697 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01490d0f-c36e-4d83-911d-8083812c3c9f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:55.379294 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01490d0f-c36e-4d83-911d-8083812c3c9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:27:55.388473 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-01490d0f-c36e-4d83-911d-8083812c3c9f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:27:55.388473 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:55.388473 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:55.388473 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:55.388473 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:55.388473 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:55.388473 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:55.388473 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:55.388473 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:55.388473 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:55.388473 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:55.388473 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:55.388473 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:27:55.388473 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:27:55.388473 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:27:55.388473 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:27:55.389410 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:27:55.389410 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:27:55.389410 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:27:55.389410 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:27:55.389410 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:55.389410 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:55.389410 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:55.389410 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:55.389410 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:55.389410 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:55.389410 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:55.389410 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:55.389410 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:55.389410 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:55.389410 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:55.389410 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:55.390271 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:55.390271 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:55.390271 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:55.390271 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:55.390271 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:27:55.390271 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:27:55.390271 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:27:55.390271 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:27:55.390271 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:27:55.390271 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:27:55.390271 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:27:55.390271 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:55.393468 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 313/616] 10.222.0.13 () {56 vars in 1102 bytes} [Fri Jun 24 08:27:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:27:55.400833 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01490d0f-c36e-4d83-911d-8083812c3c9f None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:55.445346 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf218c4-6502-4fd5-8513-de4d7e088d8c None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:55.508255 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01490d0f-c36e-4d83-911d-8083812c3c9f None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:55.531150 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc79625-df85-4e9d-b978-a8623775fd49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:55.531959 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc79625-df85-4e9d-b978-a8623775fd49 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:55.532246 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc79625-df85-4e9d-b978-a8623775fd49 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:55.532524 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc79625-df85-4e9d-b978-a8623775fd49 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:55.546243 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9800b6e0-bff1-4e09-9565-2c7701aa50ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:55.547236 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800b6e0-bff1-4e09-9565-2c7701aa50ff None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:55.547715 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800b6e0-bff1-4e09-9565-2c7701aa50ff None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:55.548202 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800b6e0-bff1-4e09-9565-2c7701aa50ff None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:55.549137 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9800b6e0-bff1-4e09-9565-2c7701aa50ff None admin] RBAC: Adding query filter params (name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:55.549632 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9800b6e0-bff1-4e09-9565-2c7701aa50ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:55.550511 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc79625-df85-4e9d-b978-a8623775fd49 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:55.551416 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9800b6e0-bff1-4e09-9565-2c7701aa50ff None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:55.552041 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc79625-df85-4e9d-b978-a8623775fd49 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:55.561438 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 314/617] 10.222.0.13 () {56 vars in 1103 bytes} [Fri Jun 24 08:27:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:55.567174 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9800b6e0-bff1-4e09-9565-2c7701aa50ff None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:55.584317 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-fdc79625-df85-4e9d-b978-a8623775fd49 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 08:27:55.584317 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:55.584317 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:55.584317 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:55.584317 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:55.584317 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:55.584317 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:55.584317 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:55.584317 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:55.584317 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:55.584317 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:55.584317 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:55.584317 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:27:55.584317 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:27:55.584317 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:27:55.584317 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:27:55.585339 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:55.585339 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:55.585339 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:27:55.585339 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:27:55.585339 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:27:55.585339 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:27:55.585339 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:55.585339 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:55.585339 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:55.585339 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:55.585339 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:55.585339 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:55.585339 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:55.585339 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:55.585339 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:55.585339 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:55.585339 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:55.585339 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:55.586946 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:55.586946 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:55.586946 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:27:55.586946 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:27:55.586946 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:27:55.586946 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:27:55.586946 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:27:55.586946 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:27:55.586946 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 08:27:55.586946 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:27:55.586946 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 304/618] 10.222.0.13 () {56 vars in 1093 bytes} [Fri Jun 24 08:27:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:27:55.592789 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc79625-df85-4e9d-b978-a8623775fd49 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:55.599614 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:55.679617 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9800b6e0-bff1-4e09-9565-2c7701aa50ff None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:55.717418 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef14054-a319-49ac-a2b1-416074c26078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:55.718706 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef14054-a319-49ac-a2b1-416074c26078 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:55.719278 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef14054-a319-49ac-a2b1-416074c26078 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:55.719762 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef14054-a319-49ac-a2b1-416074c26078 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:55.730930 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef14054-a319-49ac-a2b1-416074c26078 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:55.732731 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef14054-a319-49ac-a2b1-416074c26078 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:55.749910 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc79625-df85-4e9d-b978-a8623775fd49 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:55.752800 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-4ef14054-a319-49ac-a2b1-416074c26078 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:55.752800 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:55.752800 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:55.752800 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:55.752800 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:55.752800 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:55.752800 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:55.752800 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:55.752800 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:55.752800 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:55.752800 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:55.752800 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:55.752800 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:27:55.752800 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:27:55.752800 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:27:55.753626 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:27:55.753626 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:55.753626 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:55.753626 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:55.753626 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:55.753626 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:55.753626 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:55.753626 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:55.753626 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:55.753626 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:55.753626 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:55.753626 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:55.753626 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:55.753626 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:55.753626 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:55.753626 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:55.753626 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:55.754679 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:27:55.754679 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:27:55.754679 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:55.754679 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:55.756870 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 315/619] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:27:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:27:55.763221 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef14054-a319-49ac-a2b1-416074c26078 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:55.765040 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:55.786528 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25249de7-7ecc-4cfa-b300-1cf8e3798572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:55.787198 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25249de7-7ecc-4cfa-b300-1cf8e3798572 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:55.787482 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25249de7-7ecc-4cfa-b300-1cf8e3798572 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:55.787755 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25249de7-7ecc-4cfa-b300-1cf8e3798572 None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:55.788585 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25249de7-7ecc-4cfa-b300-1cf8e3798572 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:55.788888 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25249de7-7ecc-4cfa-b300-1cf8e3798572 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:55.797724 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25249de7-7ecc-4cfa-b300-1cf8e3798572 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:55.809307 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 305/620] 10.222.0.13 () {56 vars in 1133 bytes} [Fri Jun 24 08:27:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:27:55.814374 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25249de7-7ecc-4cfa-b300-1cf8e3798572 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:55.925557 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25249de7-7ecc-4cfa-b300-1cf8e3798572 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:55.982164 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:55.983239 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b6cf02-772c-4f0c-9859-b3a456f90f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:55.984777 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b6cf02-772c-4f0c-9859-b3a456f90f47 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:55.985934 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b6cf02-772c-4f0c-9859-b3a456f90f47 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:55.986429 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b6cf02-772c-4f0c-9859-b3a456f90f47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:56.022047 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef14054-a319-49ac-a2b1-416074c26078 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:56.072798 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:56.087942 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f02cc1-481d-4bec-b4b1-3445d3f7c19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:56.088956 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f02cc1-481d-4bec-b4b1-3445d3f7c19d None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:56.089403 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f02cc1-481d-4bec-b4b1-3445d3f7c19d None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:56.089809 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f02cc1-481d-4bec-b4b1-3445d3f7c19d None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:56.090567 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f02cc1-481d-4bec-b4b1-3445d3f7c19d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:56.090977 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f02cc1-481d-4bec-b4b1-3445d3f7c19d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:56.092520 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f02cc1-481d-4bec-b4b1-3445d3f7c19d None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:56.105773 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b6cf02-772c-4f0c-9859-b3a456f90f47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:56.106446 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 316/621] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:27:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:56.107285 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b6cf02-772c-4f0c-9859-b3a456f90f47 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:56.107846 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda707b-832f-4d43-a1f1-d7e593ef50bb None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:56.108144 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda707b-832f-4d43-a1f1-d7e593ef50bb None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:56.108422 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda707b-832f-4d43-a1f1-d7e593ef50bb None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:56.110139 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 317/622] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:56.122323 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfda707b-832f-4d43-a1f1-d7e593ef50bb None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:56.213288 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 306/623] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:27:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:27:56.214643 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f18e216-e21a-4096-8414-c47d5d9767cf None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:56.214930 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f18e216-e21a-4096-8414-c47d5d9767cf None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:56.215207 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f18e216-e21a-4096-8414-c47d5d9767cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:56.231597 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-1f18e216-e21a-4096-8414-c47d5d9767cf None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:56.232845 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-1f18e216-e21a-4096-8414-c47d5d9767cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:56.257281 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfda707b-832f-4d43-a1f1-d7e593ef50bb None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:56.306594 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a850c9-b2de-4ec9-8459-5349a8bd7474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:56.308146 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a850c9-b2de-4ec9-8459-5349a8bd7474 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:56.308682 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a850c9-b2de-4ec9-8459-5349a8bd7474 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:56.309170 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a850c9-b2de-4ec9-8459-5349a8bd7474 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:56.311410 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 307/624] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:27:56.312415 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f18e216-e21a-4096-8414-c47d5d9767cf None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:56.313534 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:56.315628 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:56.333314 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a850c9-b2de-4ec9-8459-5349a8bd7474 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:56.335728 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a850c9-b2de-4ec9-8459-5349a8bd7474 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:56.355123 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-45a850c9-b2de-4ec9-8459-5349a8bd7474 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 08:27:56.355123 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:56.355123 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:56.355123 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:56.355123 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:56.355123 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:56.355123 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:56.355123 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:56.355123 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:56.355123 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:56.355123 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:56.355123 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:56.355123 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:27:56.355123 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:27:56.355123 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:27:56.355123 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:27:56.356182 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:56.356182 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:56.356182 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:27:56.356182 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:27:56.356182 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:27:56.356182 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:27:56.356182 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:56.356182 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:56.356182 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:56.356182 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:56.356182 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:56.356182 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:56.356182 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:56.356182 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:56.356182 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:56.356182 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:56.356182 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:56.356182 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:56.357166 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:56.357166 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:56.357166 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:27:56.357166 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:27:56.357166 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:27:56.357166 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:27:56.357166 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:27:56.357166 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:27:56.357166 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 08:27:56.357166 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:56.357166 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 318/625] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Jun 24 08:27:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:27:56.357823 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a850c9-b2de-4ec9-8459-5349a8bd7474 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:56.443468 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f18e216-e21a-4096-8414-c47d5d9767cf None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:56.469964 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a850c9-b2de-4ec9-8459-5349a8bd7474 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:56.480973 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd2143a-f32f-4246-a415-7bd9340e995b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:56.481791 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd2143a-f32f-4246-a415-7bd9340e995b None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:56.482074 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd2143a-f32f-4246-a415-7bd9340e995b None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:56.482344 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd2143a-f32f-4246-a415-7bd9340e995b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:56.493380 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd2143a-f32f-4246-a415-7bd9340e995b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:56.494807 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd2143a-f32f-4246-a415-7bd9340e995b None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:56.509478 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcbae5c-262a-415d-a651-e665d7b46d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:56.510870 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcbae5c-262a-415d-a651-e665d7b46d4b None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:56.511589 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcbae5c-262a-415d-a651-e665d7b46d4b None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:56.512326 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcbae5c-262a-415d-a651-e665d7b46d4b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:56.515638 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-4fd2143a-f32f-4246-a415-7bd9340e995b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:56.515638 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:56.515638 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:56.515638 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:56.515638 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:56.515638 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:56.515638 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:56.515638 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:56.515638 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:56.515638 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:56.515638 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:56.515638 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:56.515638 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:27:56.515638 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:27:56.515638 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:27:56.516515 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:27:56.516515 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:56.516515 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:56.516515 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:56.516515 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:56.516515 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:56.516515 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:56.516515 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:56.516515 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:56.516515 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:56.516515 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:56.516515 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:56.516515 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:56.516515 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:56.516515 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:56.516515 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:56.516515 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:56.517398 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:27:56.517398 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:27:56.517398 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:56.517398 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:27:56.517398 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 308/626] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:27:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:56.518117 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd2143a-f32f-4246-a415-7bd9340e995b None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:56.532869 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcbae5c-262a-415d-a651-e665d7b46d4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:27:56.544030 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-4fcbae5c-262a-415d-a651-e665d7b46d4b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:27:56.544030 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:56.544030 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:56.544030 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:56.544030 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:56.544030 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:56.544030 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:56.544030 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:56.544030 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:56.544030 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:56.544030 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:56.544030 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:56.544030 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:27:56.544030 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:27:56.544030 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:27:56.544030 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:27:56.545011 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:27:56.545011 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:27:56.545011 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:27:56.545011 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:27:56.545011 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:56.545011 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:56.545011 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:56.545011 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:56.545011 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:56.545011 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:56.545011 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:56.545011 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:56.545011 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:56.545011 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:56.545011 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:56.545011 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:56.546809 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:56.546809 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:56.546809 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:56.546809 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:56.546809 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:27:56.546809 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:27:56.546809 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:27:56.546809 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:27:56.546809 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:27:56.546809 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:27:56.546809 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:27:56.546809 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:56.549759 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 319/627] 10.222.0.13 () {56 vars in 1102 bytes} [Fri Jun 24 08:27:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:56.550843 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcbae5c-262a-415d-a651-e665d7b46d4b None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:56.596048 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:56.635545 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:56.656871 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd2143a-f32f-4246-a415-7bd9340e995b None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:56.688172 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcbae5c-262a-415d-a651-e665d7b46d4b None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:56.705231 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf24f1ff-c75c-487e-8755-4c976c0fd1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:56.706044 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf24f1ff-c75c-487e-8755-4c976c0fd1b1 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:56.706401 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf24f1ff-c75c-487e-8755-4c976c0fd1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:56.706752 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf24f1ff-c75c-487e-8755-4c976c0fd1b1 None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:56.707661 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf24f1ff-c75c-487e-8755-4c976c0fd1b1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:56.708042 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf24f1ff-c75c-487e-8755-4c976c0fd1b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:56.709649 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf24f1ff-c75c-487e-8755-4c976c0fd1b1 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:56.733450 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 309/628] 10.222.0.13 () {56 vars in 1135 bytes} [Fri Jun 24 08:27:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:56.734861 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf24f1ff-c75c-487e-8755-4c976c0fd1b1 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:56.737317 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7177cb67-e749-4bec-a374-949c983ae93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:56.738109 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7177cb67-e749-4bec-a374-949c983ae93f None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:56.738487 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7177cb67-e749-4bec-a374-949c983ae93f None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:56.738842 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7177cb67-e749-4bec-a374-949c983ae93f None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:56.740269 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7177cb67-e749-4bec-a374-949c983ae93f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:56.740269 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7177cb67-e749-4bec-a374-949c983ae93f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:56.741450 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7177cb67-e749-4bec-a374-949c983ae93f None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:56.776998 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 320/629] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:27:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:56.777417 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7177cb67-e749-4bec-a374-949c983ae93f None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:56.850524 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf24f1ff-c75c-487e-8755-4c976c0fd1b1 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:56.870585 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:56.902642 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d6b209-f423-4f43-834e-fa8a78d9d74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:56.903653 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d6b209-f423-4f43-834e-fa8a78d9d74d None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:56.904111 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d6b209-f423-4f43-834e-fa8a78d9d74d None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:56.904530 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7177cb67-e749-4bec-a374-949c983ae93f None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:56.904810 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d6b209-f423-4f43-834e-fa8a78d9d74d None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:56.905764 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d6b209-f423-4f43-834e-fa8a78d9d74d None admin] RBAC: Adding query filter params (name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:56.906502 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d6b209-f423-4f43-834e-fa8a78d9d74d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:56.908990 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d6b209-f423-4f43-834e-fa8a78d9d74d None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:56.925324 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 310/630] 10.222.0.13 () {56 vars in 1103 bytes} [Fri Jun 24 08:27:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:56.926934 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:56.927115 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d6b209-f423-4f43-834e-fa8a78d9d74d None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:56.948882 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d85491-e78f-4f93-abe8-273aebaffcca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:56.949821 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d85491-e78f-4f93-abe8-273aebaffcca None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:56.950394 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d85491-e78f-4f93-abe8-273aebaffcca None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:56.950512 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d85491-e78f-4f93-abe8-273aebaffcca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:56.962099 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d85491-e78f-4f93-abe8-273aebaffcca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:56.963819 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d85491-e78f-4f93-abe8-273aebaffcca None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:56.982349 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-93d85491-e78f-4f93-abe8-273aebaffcca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:56.982349 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:56.982349 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:56.982349 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:56.982349 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:56.982349 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:56.982349 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:56.982349 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:56.982349 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:56.982349 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:56.982349 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:56.982349 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:56.982349 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:27:56.982349 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:27:56.982349 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:27:56.983304 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:27:56.983304 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:56.983304 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:56.983304 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:56.983304 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:56.983304 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:56.983304 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:56.983304 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:56.983304 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:56.983304 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:56.983304 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:56.983304 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:56.983304 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:56.983304 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:56.983304 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:56.983304 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:56.983304 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:56.984197 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:27:56.984197 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:27:56.984197 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:56.984197 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:56.988415 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 321/631] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:27:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:56.989367 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d85491-e78f-4f93-abe8-273aebaffcca None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:57.064831 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d6b209-f423-4f43-834e-fa8a78d9d74d None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:57.102924 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d85491-e78f-4f93-abe8-273aebaffcca None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:57.113209 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f915656-d219-494d-8a67-424a65c84df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:57.114586 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f915656-d219-494d-8a67-424a65c84df6 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:57.114878 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f915656-d219-494d-8a67-424a65c84df6 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:57.115158 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f915656-d219-494d-8a67-424a65c84df6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:57.123229 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f915656-d219-494d-8a67-424a65c84df6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:57.124691 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f915656-d219-494d-8a67-424a65c84df6 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:57.140163 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-7f915656-d219-494d-8a67-424a65c84df6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:27:57.140163 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:57.140163 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:57.140163 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:57.140163 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:57.140163 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:57.140163 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:57.140163 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:57.140163 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:57.140163 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:57.140163 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:57.140163 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:57.140163 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:27:57.140163 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:27:57.140163 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:27:57.141106 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:27:57.141106 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:57.141106 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:57.141106 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:57.141106 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:57.141106 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:57.141106 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:57.141106 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:57.141106 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:57.141106 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:57.141106 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:57.141106 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:57.141106 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:57.141106 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:57.141106 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:57.141106 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:57.141106 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:57.142004 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:27:57.142004 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:27:57.142004 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:27:57.142004 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:27:57.142004 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:27:57.142004 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:27:57.142004 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:27:57.142004 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:27:57.142004 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 311/632] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:27:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:57.143052 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f915656-d219-494d-8a67-424a65c84df6 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:57.150561 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d8bc73-0740-4586-819b-cec6622aa548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:57.151489 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d8bc73-0740-4586-819b-cec6622aa548 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:57.151911 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d8bc73-0740-4586-819b-cec6622aa548 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:57.152662 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d8bc73-0740-4586-819b-cec6622aa548 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:57.164167 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d8bc73-0740-4586-819b-cec6622aa548 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:57.166512 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d8bc73-0740-4586-819b-cec6622aa548 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:57.178088 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:57.185895 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-03d8bc73-0740-4586-819b-cec6622aa548 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:57.185895 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:57.185895 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:57.185895 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:57.185895 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:57.185895 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:57.185895 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:57.185895 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:57.185895 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:57.185895 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:57.185895 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:57.185895 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:57.185895 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:27:57.185895 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:27:57.185895 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:27:57.186991 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:27:57.186991 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:57.186991 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:57.186991 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:57.186991 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:57.186991 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:57.186991 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:57.186991 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:57.186991 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:57.186991 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:57.186991 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:57.186991 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:57.186991 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:57.186991 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:57.186991 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:57.186991 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:57.186991 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:57.188065 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:27:57.188065 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:27:57.188065 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:57.188065 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:57.191960 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 322/633] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:27:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:57.193209 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d0c3fa-a48e-4198-b454-b04fc45d9798 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:57.193493 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d0c3fa-a48e-4198-b454-b04fc45d9798 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:57.193783 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d0c3fa-a48e-4198-b454-b04fc45d9798 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:57.195058 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 323/634] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:57.197864 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d0c3fa-a48e-4198-b454-b04fc45d9798 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:57.219517 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:57.258526 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f915656-d219-494d-8a67-424a65c84df6 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:57.295090 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ee1ab-c327-4b22-a843-62cfc3616a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:57.295742 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ee1ab-c327-4b22-a843-62cfc3616a78 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:57.296027 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ee1ab-c327-4b22-a843-62cfc3616a78 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:57.296307 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ee1ab-c327-4b22-a843-62cfc3616a78 None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:57.297026 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76ee1ab-c327-4b22-a843-62cfc3616a78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:57.297342 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76ee1ab-c327-4b22-a843-62cfc3616a78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:57.298769 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76ee1ab-c327-4b22-a843-62cfc3616a78 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:57.307412 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d0c3fa-a48e-4198-b454-b04fc45d9798 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:57.315424 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 312/635] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:27:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:57.315424 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ee1ab-c327-4b22-a843-62cfc3616a78 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:57.343559 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ad799-8fea-41df-a836-c785945d3a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:57.344213 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ad799-8fea-41df-a836-c785945d3a67 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:57.344491 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ad799-8fea-41df-a836-c785945d3a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:57.344767 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ad799-8fea-41df-a836-c785945d3a67 None admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:57.345545 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51ad799-8fea-41df-a836-c785945d3a67 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:57.345883 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51ad799-8fea-41df-a836-c785945d3a67 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:57.347326 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51ad799-8fea-41df-a836-c785945d3a67 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:57.359366 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 324/636] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Jun 24 08:27:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:57.360595 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e6ee5-d04c-4f7d-9bec-c541592f6d9f None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:57.360895 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e6ee5-d04c-4f7d-9bec-c541592f6d9f None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:57.361159 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e6ee5-d04c-4f7d-9bec-c541592f6d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:57.379336 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-d04e6ee5-d04c-4f7d-9bec-c541592f6d9f None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:57.380521 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-d04e6ee5-d04c-4f7d-9bec-c541592f6d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:57.408870 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:57.417041 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ee1ab-c327-4b22-a843-62cfc3616a78 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:57.447406 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 325/637] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:27:57.448851 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa5246-d2a6-49ca-9bb0-aa0b51826a9c None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:57.449143 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa5246-d2a6-49ca-9bb0-aa0b51826a9c None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:57.449418 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa5246-d2a6-49ca-9bb0-aa0b51826a9c None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:57.450850 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 326/638] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:57.451463 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fa5246-d2a6-49ca-9bb0-aa0b51826a9c None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:57.453278 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60e6a30-9d81-422f-8d36-908835264b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:57.453975 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e6a30-9d81-422f-8d36-908835264b43 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:57.454256 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e6a30-9d81-422f-8d36-908835264b43 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:57.454534 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e6a30-9d81-422f-8d36-908835264b43 None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:57.455251 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60e6a30-9d81-422f-8d36-908835264b43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:57.455553 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60e6a30-9d81-422f-8d36-908835264b43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:57.456922 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60e6a30-9d81-422f-8d36-908835264b43 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:57.468842 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:57.474034 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 313/639] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:27:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:57.474924 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60e6a30-9d81-422f-8d36-908835264b43 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:57.563187 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fa5246-d2a6-49ca-9bb0-aa0b51826a9c None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:57.574384 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60e6a30-9d81-422f-8d36-908835264b43 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:57.595846 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae3e413-e938-4289-aff8-7c11071bc5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:57.596628 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3e413-e938-4289-aff8-7c11071bc5f9 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:57.596902 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3e413-e938-4289-aff8-7c11071bc5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:57.597175 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3e413-e938-4289-aff8-7c11071bc5f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:57.604711 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae3e413-e938-4289-aff8-7c11071bc5f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:57.606191 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae3e413-e938-4289-aff8-7c11071bc5f9 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:57.607648 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d67bb-ed36-4102-8005-f9acf3e26412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:57.608290 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d67bb-ed36-4102-8005-f9acf3e26412 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:57.608565 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d67bb-ed36-4102-8005-f9acf3e26412 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:57.608847 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d67bb-ed36-4102-8005-f9acf3e26412 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:57.609666 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12d67bb-ed36-4102-8005-f9acf3e26412 None admin] RBAC: Adding query filter params (role.id=6ca1ae4292d9414ca6526c6e8608de9e, scope.project.id=180b0af5863349e99ae412b07c1e9465, user.id=74bb1bc6821b4500b26f160ccae5b7d5) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:57.610005 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12d67bb-ed36-4102-8005-f9acf3e26412 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:57.611418 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12d67bb-ed36-4102-8005-f9acf3e26412 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:57.617245 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 314/640] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Jun 24 08:27:57 2022] GET /identity/v3/role_assignments?user.id=74bb1bc6821b4500b26f160ccae5b7d5&scope.project.id=180b0af5863349e99ae412b07c1e9465&role.id=6ca1ae4292d9414ca6526c6e8608de9e => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:57.618897 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6393bf-86d4-41b9-afd4-4be105a35732 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:57.619251 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6393bf-86d4-41b9-afd4-4be105a35732 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:57.619538 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6393bf-86d4-41b9-afd4-4be105a35732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:57.620633 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-3ae3e413-e938-4289-aff8-7c11071bc5f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:27:57.620633 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:57.620633 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:57.620633 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:57.620633 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:57.620633 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:57.620633 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:57.620633 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:57.620633 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:57.620633 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:57.620633 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:57.620633 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:57.620633 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:27:57.620633 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:27:57.620633 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:27:57.621464 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:27:57.621464 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:57.621464 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:57.621464 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:57.621464 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:57.621464 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:57.621464 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:57.621464 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:57.621464 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:57.621464 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:57.621464 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:57.621464 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:57.621464 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:57.621464 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:57.621464 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:57.621464 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:57.621464 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:57.622375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:27:57.622375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:27:57.622375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:27:57.622375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:27:57.622375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:27:57.622375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:27:57.622375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:27:57.622375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:57.622375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 327/641] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:27:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:57.622854 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae3e413-e938-4289-aff8-7c11071bc5f9 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:57.641600 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-9f6393bf-86d4-41b9-afd4-4be105a35732 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:57.642929 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-9f6393bf-86d4-41b9-afd4-4be105a35732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:57.659658 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:57.713933 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:57.726968 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 315/642] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:27:57.729003 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6393bf-86d4-41b9-afd4-4be105a35732 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:57.751527 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae3e413-e938-4289-aff8-7c11071bc5f9 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:57.792467 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd6458-ffe7-4085-ab91-2224f36a00bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:57.793282 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd6458-ffe7-4085-ab91-2224f36a00bc None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:57.793570 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd6458-ffe7-4085-ab91-2224f36a00bc None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:57.793896 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd6458-ffe7-4085-ab91-2224f36a00bc None admin] PATH_INFO: `/v3/users/glance` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:57.802985 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fd6458-ffe7-4085-ab91-2224f36a00bc None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:57.803805 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fd6458-ffe7-4085-ab91-2224f36a00bc None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:57.819416 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-c9fd6458-ffe7-4085-ab91-2224f36a00bc None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 24 08:27:57.819416 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:57.819416 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:57.819416 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:57.819416 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:57.819416 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:57.819416 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:57.819416 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:57.819416 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:57.819416 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:57.819416 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:57.819416 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:57.819416 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:27:57.819416 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:27:57.819416 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:27:57.819416 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:27:57.820599 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:57.820599 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:57.820599 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:27:57.820599 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:27:57.820599 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:27:57.820599 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:27:57.820599 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:57.820599 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:57.820599 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:57.820599 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:57.820599 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:57.820599 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:57.820599 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:57.820599 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:57.820599 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:57.820599 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:57.820599 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:57.820599 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:57.822767 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:57.822767 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:57.822767 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:27:57.822767 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:27:57.822767 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:27:57.822767 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:27:57.822767 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:27:57.822767 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:27:57.822767 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 24 08:27:57.822767 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:27:57.822767 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 328/643] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:27:57 2022] GET /identity/v3/users/glance => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:57.822767 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd6458-ffe7-4085-ab91-2224f36a00bc None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:57.857460 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6393bf-86d4-41b9-afd4-4be105a35732 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:57.896482 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da75427-f555-4dc2-bbec-51e91a22ce9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:57.897523 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da75427-f555-4dc2-bbec-51e91a22ce9e None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:57.897999 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da75427-f555-4dc2-bbec-51e91a22ce9e None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:57.898534 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da75427-f555-4dc2-bbec-51e91a22ce9e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:57.911794 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da75427-f555-4dc2-bbec-51e91a22ce9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:27:57.920247 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-4da75427-f555-4dc2-bbec-51e91a22ce9e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:27:57.920247 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:57.920247 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:57.920247 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:57.920247 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:57.920247 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:57.920247 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:57.920247 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:57.920247 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:57.920247 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:57.920247 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:57.920247 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:57.920247 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:27:57.920247 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:27:57.920247 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:27:57.920247 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:27:57.921344 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:27:57.921344 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:27:57.921344 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:27:57.921344 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:27:57.921344 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:57.921344 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:57.921344 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:57.921344 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:57.921344 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:57.921344 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:57.921344 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:57.921344 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:57.921344 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:57.921344 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:57.921344 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:57.921344 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:57.922535 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:57.922535 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:57.922535 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:57.922535 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:57.922535 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:27:57.922535 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:27:57.922535 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:27:57.922535 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:27:57.922535 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:27:57.922535 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:27:57.922535 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:27:57.922535 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:27:57.922535 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 316/644] 10.222.0.13 () {56 vars in 1102 bytes} [Fri Jun 24 08:27:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:57.922535 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da75427-f555-4dc2-bbec-51e91a22ce9e None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:57.934380 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:57.950299 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd6458-ffe7-4085-ab91-2224f36a00bc None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:57.980099 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:57.988300 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa58b92b-be9c-4936-9ffe-455c6ef4324a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:57.989368 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa58b92b-be9c-4936-9ffe-455c6ef4324a None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:57.989785 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa58b92b-be9c-4936-9ffe-455c6ef4324a None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:57.990318 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa58b92b-be9c-4936-9ffe-455c6ef4324a None admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:57.991514 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa58b92b-be9c-4936-9ffe-455c6ef4324a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:57.992142 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa58b92b-be9c-4936-9ffe-455c6ef4324a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:57.994896 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa58b92b-be9c-4936-9ffe-455c6ef4324a None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:58.016726 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 329/645] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Jun 24 08:27:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:58.016726 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa58b92b-be9c-4936-9ffe-455c6ef4324a None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:58.035129 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da75427-f555-4dc2-bbec-51e91a22ce9e None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:58.070192 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675d7cc7-4b25-4bcc-8e40-e7147e3cf4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:58.071616 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d7cc7-4b25-4bcc-8e40-e7147e3cf4e9 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:58.071895 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d7cc7-4b25-4bcc-8e40-e7147e3cf4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:58.072323 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d7cc7-4b25-4bcc-8e40-e7147e3cf4e9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:58.082717 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675d7cc7-4b25-4bcc-8e40-e7147e3cf4e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:27:58.090728 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-675d7cc7-4b25-4bcc-8e40-e7147e3cf4e9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:27:58.090728 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:58.090728 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:58.090728 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:58.090728 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:58.090728 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:58.090728 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:58.090728 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:58.090728 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:58.090728 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:58.090728 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:58.090728 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:58.090728 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:27:58.090728 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:27:58.090728 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:27:58.090728 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:27:58.091669 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:27:58.091669 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:27:58.091669 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:27:58.091669 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:27:58.091669 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:58.091669 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:58.091669 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:58.091669 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:58.091669 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:58.091669 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:58.091669 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:58.091669 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:58.091669 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:58.091669 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:58.091669 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:58.091669 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:58.092534 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:58.092534 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:58.092534 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:58.092534 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:58.092534 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:27:58.092534 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:27:58.092534 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:27:58.092534 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:27:58.092534 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:27:58.092534 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:27:58.092534 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:27:58.092534 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:27:58.092534 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 317/646] 10.222.0.13 () {56 vars in 1102 bytes} [Fri Jun 24 08:27:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:58.093259 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675d7cc7-4b25-4bcc-8e40-e7147e3cf4e9 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:58.132933 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa58b92b-be9c-4936-9ffe-455c6ef4324a None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:58.170853 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:58.172735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736d2686-4716-4523-8747-280056fe5c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:58.173403 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736d2686-4716-4523-8747-280056fe5c53 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:58.173683 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736d2686-4716-4523-8747-280056fe5c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:58.173981 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736d2686-4716-4523-8747-280056fe5c53 None admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:58.174842 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736d2686-4716-4523-8747-280056fe5c53 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:58.175143 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736d2686-4716-4523-8747-280056fe5c53 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:58.176562 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736d2686-4716-4523-8747-280056fe5c53 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:58.199687 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 330/647] 10.222.0.13 () {56 vars in 1137 bytes} [Fri Jun 24 08:27:58 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:58.199687 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736d2686-4716-4523-8747-280056fe5c53 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:58.213642 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675d7cc7-4b25-4bcc-8e40-e7147e3cf4e9 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:58.246044 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:58.252983 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6acd6d9-be1c-4831-9937-52a3cbfc2981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:58.253835 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6acd6d9-be1c-4831-9937-52a3cbfc2981 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:58.254118 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6acd6d9-be1c-4831-9937-52a3cbfc2981 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:58.254392 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6acd6d9-be1c-4831-9937-52a3cbfc2981 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:58.255463 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6acd6d9-be1c-4831-9937-52a3cbfc2981 None admin] RBAC: Adding query filter params (name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:58.256034 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6acd6d9-be1c-4831-9937-52a3cbfc2981 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:58.257964 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6acd6d9-be1c-4831-9937-52a3cbfc2981 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:58.278845 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 318/648] 10.222.0.13 () {56 vars in 1103 bytes} [Fri Jun 24 08:27:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:58.278845 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6acd6d9-be1c-4831-9937-52a3cbfc2981 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:58.306891 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736d2686-4716-4523-8747-280056fe5c53 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:58.344758 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8ff633-6f4a-4c1f-b16a-b5f201896cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:58.345739 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ff633-6f4a-4c1f-b16a-b5f201896cc6 None admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:58.345934 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ff633-6f4a-4c1f-b16a-b5f201896cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:58.346203 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ff633-6f4a-4c1f-b16a-b5f201896cc6 None admin] PATH_INFO: `/v3/projects/180b0af5863349e99ae412b07c1e9465/users/bec8c2f113cf41a0939b81f38fbee813/roles/6ca1ae4292d9414ca6526c6e8608de9e` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:58.364409 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8ff633-6f4a-4c1f-b16a-b5f201896cc6 None admin] RBAC: Authorizing `identity:create_grant(project_id=180b0af5863349e99ae412b07c1e9465, user_id=bec8c2f113cf41a0939b81f38fbee813, role_id=6ca1ae4292d9414ca6526c6e8608de9e)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:58.365875 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8ff633-6f4a-4c1f-b16a-b5f201896cc6 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:58.393821 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6acd6d9-be1c-4831-9937-52a3cbfc2981 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:58.394033 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 331/649] 10.222.0.13 () {58 vars in 1439 bytes} [Fri Jun 24 08:27:58 2022] PUT /identity/v3/projects/180b0af5863349e99ae412b07c1e9465/users/bec8c2f113cf41a0939b81f38fbee813/roles/6ca1ae4292d9414ca6526c6e8608de9e => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:27:58.394033 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8ff633-6f4a-4c1f-b16a-b5f201896cc6 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:58.422597 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:58.431005 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343936f4-5bcc-4b31-ae88-11f59035e611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:58.431829 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343936f4-5bcc-4b31-ae88-11f59035e611 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:58.432181 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343936f4-5bcc-4b31-ae88-11f59035e611 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:58.432573 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343936f4-5bcc-4b31-ae88-11f59035e611 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:58.433394 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343936f4-5bcc-4b31-ae88-11f59035e611 None admin] RBAC: Adding query filter params (name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:58.433842 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343936f4-5bcc-4b31-ae88-11f59035e611 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:58.435336 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343936f4-5bcc-4b31-ae88-11f59035e611 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:58.451244 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 319/650] 10.222.0.13 () {56 vars in 1103 bytes} [Fri Jun 24 08:27:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:58.452315 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343936f4-5bcc-4b31-ae88-11f59035e611 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:58.545884 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:58.552469 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8ff633-6f4a-4c1f-b16a-b5f201896cc6 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:58.578664 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343936f4-5bcc-4b31-ae88-11f59035e611 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:58.593270 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17753cd3-a4f3-4baf-a7f6-10997e09c9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:58.594302 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17753cd3-a4f3-4baf-a7f6-10997e09c9c2 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:58.594798 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17753cd3-a4f3-4baf-a7f6-10997e09c9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:58.595266 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17753cd3-a4f3-4baf-a7f6-10997e09c9c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:58.629498 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6fbedb-3094-4e2d-82f4-a938ee11fb5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:58.631289 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6fbedb-3094-4e2d-82f4-a938ee11fb5a None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:58.632074 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6fbedb-3094-4e2d-82f4-a938ee11fb5a None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:58.632550 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6fbedb-3094-4e2d-82f4-a938ee11fb5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:58.647426 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6fbedb-3094-4e2d-82f4-a938ee11fb5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:58.649885 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6fbedb-3094-4e2d-82f4-a938ee11fb5a None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:58.673330 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-2d6fbedb-3094-4e2d-82f4-a938ee11fb5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:58.673330 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:58.673330 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:58.673330 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:58.673330 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:58.673330 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:58.673330 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:58.673330 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:58.673330 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:58.673330 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:58.673330 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:58.673330 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:58.673330 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:27:58.673330 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:27:58.673330 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:27:58.674481 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:27:58.674481 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:58.674481 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:58.674481 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:58.674481 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:58.674481 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:58.674481 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:58.674481 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:58.674481 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:58.674481 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:58.674481 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:58.674481 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:58.674481 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:58.674481 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:58.674481 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:58.674481 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:58.674481 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:58.675345 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:27:58.675345 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:27:58.675345 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:58.675345 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:27:58.675557 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17753cd3-a4f3-4baf-a7f6-10997e09c9c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:58.675557 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17753cd3-a4f3-4baf-a7f6-10997e09c9c2 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:58.678935 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 320/651] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:27:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:58.679945 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6fbedb-3094-4e2d-82f4-a938ee11fb5a None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:58.736688 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:58.800813 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6fbedb-3094-4e2d-82f4-a938ee11fb5a None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:58.818278 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 332/652] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:27:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:27:58.820103 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17753cd3-a4f3-4baf-a7f6-10997e09c9c2 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:58.839241 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e8a14-512d-4095-a2e0-0b074f9912ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:58.840152 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e8a14-512d-4095-a2e0-0b074f9912ac None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:58.840500 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e8a14-512d-4095-a2e0-0b074f9912ac None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:58.840865 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e8a14-512d-4095-a2e0-0b074f9912ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:58.850067 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:58.853135 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8e8a14-512d-4095-a2e0-0b074f9912ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:58.854746 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8e8a14-512d-4095-a2e0-0b074f9912ac None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:58.874553 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-7c8e8a14-512d-4095-a2e0-0b074f9912ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:58.874553 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:58.874553 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:58.874553 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:58.874553 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:58.874553 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:58.874553 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:58.874553 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:58.874553 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:58.874553 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:58.874553 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:58.874553 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:58.874553 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:27:58.874553 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:27:58.874553 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:27:58.875503 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:27:58.875503 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:58.875503 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:58.875503 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:58.875503 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:58.875503 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:58.875503 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:58.875503 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:58.875503 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:58.875503 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:58.875503 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:58.875503 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:58.875503 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:58.875503 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:58.875503 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:58.875503 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:58.875503 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:58.877082 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:27:58.877082 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:27:58.877082 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:58.877082 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:27:58.877082 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 321/653] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:27:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:58.877370 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e8a14-512d-4095-a2e0-0b074f9912ac None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:58.948901 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17753cd3-a4f3-4baf-a7f6-10997e09c9c2 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:58.980215 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:58.985393 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7e642d-60c0-437f-8090-234685159c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:58.986161 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e642d-60c0-437f-8090-234685159c6b None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:58.986448 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e642d-60c0-437f-8090-234685159c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:58.986771 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e642d-60c0-437f-8090-234685159c6b None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:58.987500 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7e642d-60c0-437f-8090-234685159c6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:58.987805 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7e642d-60c0-437f-8090-234685159c6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:58.988207 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e8a14-512d-4095-a2e0-0b074f9912ac None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:58.989170 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7e642d-60c0-437f-8090-234685159c6b None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:59.006117 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 333/654] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:27:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:59.006988 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7e642d-60c0-437f-8090-234685159c6b None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:59.025439 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b345998-27f6-482c-8a43-6fd35ed44786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:59.026548 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b345998-27f6-482c-8a43-6fd35ed44786 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:59.027008 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b345998-27f6-482c-8a43-6fd35ed44786 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:59.027455 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b345998-27f6-482c-8a43-6fd35ed44786 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:59.039174 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b345998-27f6-482c-8a43-6fd35ed44786 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:59.041513 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b345998-27f6-482c-8a43-6fd35ed44786 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:59.059230 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-7b345998-27f6-482c-8a43-6fd35ed44786 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:59.059230 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:59.059230 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:59.059230 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:59.059230 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:59.059230 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:59.059230 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:59.059230 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:59.059230 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:59.059230 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:59.059230 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:59.059230 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:59.059230 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:27:59.059230 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:27:59.059230 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:27:59.060145 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:27:59.060145 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:59.060145 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:59.060145 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:59.060145 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:59.060145 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:59.060145 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:59.060145 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:59.060145 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:59.060145 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:59.060145 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:59.060145 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:59.060145 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:59.060145 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:59.060145 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:59.060145 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:59.060145 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:59.061027 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:27:59.061027 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:27:59.061027 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:59.061027 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:27:59.063344 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 322/655] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:27:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:59.064360 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b345998-27f6-482c-8a43-6fd35ed44786 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:59.071679 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:59.116916 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7e642d-60c0-437f-8090-234685159c6b None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:59.154985 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc9e74f-50ca-42fe-9725-3bc0af2f08c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:59.155696 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9e74f-50ca-42fe-9725-3bc0af2f08c8 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:59.155985 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9e74f-50ca-42fe-9725-3bc0af2f08c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:59.156261 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9e74f-50ca-42fe-9725-3bc0af2f08c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:59.156982 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc9e74f-50ca-42fe-9725-3bc0af2f08c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:59.157289 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc9e74f-50ca-42fe-9725-3bc0af2f08c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:59.158695 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc9e74f-50ca-42fe-9725-3bc0af2f08c8 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:59.172124 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 334/656] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:27:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:59.172966 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc9e74f-50ca-42fe-9725-3bc0af2f08c8 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:59.174851 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b345998-27f6-482c-8a43-6fd35ed44786 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:59.210633 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1068a20d-6e0e-4381-8f1e-63f39a3a7b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:59.212262 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068a20d-6e0e-4381-8f1e-63f39a3a7b38 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:59.212720 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068a20d-6e0e-4381-8f1e-63f39a3a7b38 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:59.213179 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068a20d-6e0e-4381-8f1e-63f39a3a7b38 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:59.221388 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1068a20d-6e0e-4381-8f1e-63f39a3a7b38 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:59.223130 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1068a20d-6e0e-4381-8f1e-63f39a3a7b38 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:59.237804 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-1068a20d-6e0e-4381-8f1e-63f39a3a7b38 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 08:27:59.237804 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:59.237804 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:59.237804 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:59.237804 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:59.237804 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:59.237804 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:59.237804 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:59.237804 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:59.237804 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:59.237804 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:59.237804 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:59.237804 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:27:59.237804 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:27:59.237804 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:27:59.237804 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:27:59.238767 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:59.238767 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:59.238767 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:27:59.238767 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:27:59.238767 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:27:59.238767 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:27:59.238767 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:59.238767 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:59.238767 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:59.238767 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:59.238767 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:59.238767 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:59.238767 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:59.238767 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:59.238767 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:59.238767 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:59.238767 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:59.238767 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:59.239663 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:59.239663 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:59.239663 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:27:59.239663 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:27:59.239663 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:27:59.239663 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:27:59.239663 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:27:59.239663 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:27:59.239663 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 08:27:59.239663 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:27:59.242336 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 323/657] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:27:59 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:59.243830 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1068a20d-6e0e-4381-8f1e-63f39a3a7b38 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:59.256914 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:59.292469 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc9e74f-50ca-42fe-9725-3bc0af2f08c8 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:59.312155 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:59.332154 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aba1d5-f55f-458a-8e1e-aa2fbf2f43a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:59.332825 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aba1d5-f55f-458a-8e1e-aa2fbf2f43a8 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:59.333112 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aba1d5-f55f-458a-8e1e-aa2fbf2f43a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:59.333392 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aba1d5-f55f-458a-8e1e-aa2fbf2f43a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:59.334161 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aba1d5-f55f-458a-8e1e-aa2fbf2f43a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:59.334470 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aba1d5-f55f-458a-8e1e-aa2fbf2f43a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:59.335852 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aba1d5-f55f-458a-8e1e-aa2fbf2f43a8 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:59.348743 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 335/658] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:27:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:59.349632 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aba1d5-f55f-458a-8e1e-aa2fbf2f43a8 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:59.359221 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1068a20d-6e0e-4381-8f1e-63f39a3a7b38 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:59.399532 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31219be-5730-4ccb-8181-80b08295c0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:59.400492 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31219be-5730-4ccb-8181-80b08295c0ee None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:59.400847 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31219be-5730-4ccb-8181-80b08295c0ee None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:59.401186 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31219be-5730-4ccb-8181-80b08295c0ee None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:59.409288 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31219be-5730-4ccb-8181-80b08295c0ee None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:59.410901 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31219be-5730-4ccb-8181-80b08295c0ee None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:59.425911 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-b31219be-5730-4ccb-8181-80b08295c0ee None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 08:27:59.425911 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:59.425911 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:59.425911 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:59.425911 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:59.425911 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:59.425911 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:59.425911 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:59.425911 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:59.425911 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:59.425911 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:59.425911 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:59.425911 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:27:59.425911 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:27:59.425911 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:27:59.425911 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:27:59.427103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:59.427103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:59.427103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:27:59.427103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:27:59.427103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:27:59.427103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:27:59.427103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:59.427103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:59.427103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:59.427103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:59.427103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:59.427103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:59.427103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:59.427103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:59.427103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:59.427103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:59.427103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:59.427103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:59.428269 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:59.428269 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:59.428269 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:27:59.428269 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:27:59.428269 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:27:59.428269 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:27:59.428269 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:27:59.428269 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:27:59.428269 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 08:27:59.428269 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:27:59.429276 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 324/659] 10.222.0.13 () {56 vars in 1102 bytes} [Fri Jun 24 08:27:59 2022] GET /identity/v3/users/neutron => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:59.430870 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31219be-5730-4ccb-8181-80b08295c0ee None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:59.469002 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aba1d5-f55f-458a-8e1e-aa2fbf2f43a8 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:59.500951 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:59.508687 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95891581-11b1-4e05-aefa-7d1264255caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:59.509366 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95891581-11b1-4e05-aefa-7d1264255caf None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:59.509646 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95891581-11b1-4e05-aefa-7d1264255caf None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:59.509942 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95891581-11b1-4e05-aefa-7d1264255caf None admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:59.510792 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95891581-11b1-4e05-aefa-7d1264255caf None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:59.511544 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95891581-11b1-4e05-aefa-7d1264255caf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:59.513329 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95891581-11b1-4e05-aefa-7d1264255caf None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:59.537175 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31219be-5730-4ccb-8181-80b08295c0ee None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:59.546834 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 336/660] 10.222.0.13 () {56 vars in 1137 bytes} [Fri Jun 24 08:27:59 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:59.548191 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95891581-11b1-4e05-aefa-7d1264255caf None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:59.586169 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd2d6f3-4433-4504-b0c4-c1c15be2c6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:59.588587 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd2d6f3-4433-4504-b0c4-c1c15be2c6d4 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:59.589163 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd2d6f3-4433-4504-b0c4-c1c15be2c6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:59.589933 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:59.590847 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd2d6f3-4433-4504-b0c4-c1c15be2c6d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:59.598964 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd2d6f3-4433-4504-b0c4-c1c15be2c6d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:59.601088 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd2d6f3-4433-4504-b0c4-c1c15be2c6d4 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:59.618822 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-2fd2d6f3-4433-4504-b0c4-c1c15be2c6d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:27:59.618822 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:59.618822 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:59.618822 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:59.618822 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:59.618822 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:59.618822 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:59.618822 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:59.618822 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:59.618822 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:59.618822 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:59.618822 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:59.618822 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:27:59.618822 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:27:59.618822 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:27:59.619740 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:27:59.619740 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:59.619740 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:59.619740 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:59.619740 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:59.619740 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:59.619740 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:59.619740 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:59.619740 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:59.619740 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:59.619740 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:59.619740 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:59.619740 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:59.619740 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:59.619740 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:59.619740 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:59.619740 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:59.620736 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:27:59.620736 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:27:59.620736 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:27:59.620736 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:27:59.620736 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:27:59.620736 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:27:59.620736 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:27:59.620736 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:27:59.623470 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 325/661] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:27:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:59.624586 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd2d6f3-4433-4504-b0c4-c1c15be2c6d4 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:59.659300 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95891581-11b1-4e05-aefa-7d1264255caf None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:59.694546 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fc255b-3c25-444b-8a8b-fb744b3c3463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:59.695333 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc255b-3c25-444b-8a8b-fb744b3c3463 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:59.696182 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc255b-3c25-444b-8a8b-fb744b3c3463 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:59.697289 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc255b-3c25-444b-8a8b-fb744b3c3463 None admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:59.699859 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fc255b-3c25-444b-8a8b-fb744b3c3463 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:59.700186 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fc255b-3c25-444b-8a8b-fb744b3c3463 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:59.701638 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fc255b-3c25-444b-8a8b-fb744b3c3463 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:59.721985 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 337/662] 10.222.0.13 () {56 vars in 1139 bytes} [Fri Jun 24 08:27:59 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:59.722812 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fc255b-3c25-444b-8a8b-fb744b3c3463 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:59.739027 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd2d6f3-4433-4504-b0c4-c1c15be2c6d4 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:59.769083 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:59.779122 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d18772-57ee-4a2d-8b67-d3b5438860c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:59.780074 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d18772-57ee-4a2d-8b67-d3b5438860c2 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:59.780563 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d18772-57ee-4a2d-8b67-d3b5438860c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:59.781135 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d18772-57ee-4a2d-8b67-d3b5438860c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:59.822552 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:59.829185 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fc255b-3c25-444b-8a8b-fb744b3c3463 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:59.858332 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d18772-57ee-4a2d-8b67-d3b5438860c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:59.860126 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d18772-57ee-4a2d-8b67-d3b5438860c2 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:59.869798 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a76e3-105e-4528-8d31-a78f9b03764f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:59.870465 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a76e3-105e-4528-8d31-a78f9b03764f None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:59.870750 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a76e3-105e-4528-8d31-a78f9b03764f None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:59.871024 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a76e3-105e-4528-8d31-a78f9b03764f None admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:59.871844 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106a76e3-105e-4528-8d31-a78f9b03764f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:59.872154 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106a76e3-105e-4528-8d31-a78f9b03764f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:59.873571 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106a76e3-105e-4528-8d31-a78f9b03764f None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:59.886181 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 338/663] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Jun 24 08:27:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:59.887058 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a76e3-105e-4528-8d31-a78f9b03764f None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:59.985356 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 326/664] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:27:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:27:59.985356 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb7fe4f-93df-4101-a0a9-3df012beb621 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:59.985356 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb7fe4f-93df-4101-a0a9-3df012beb621 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:59.985356 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb7fe4f-93df-4101-a0a9-3df012beb621 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:59.985713 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 327/665] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:27:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:59.986440 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb7fe4f-93df-4101-a0a9-3df012beb621 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:00.008477 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a76e3-105e-4528-8d31-a78f9b03764f None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:00.017758 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:00.047959 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbbc437-c33b-4127-8be9-d6d4cbe480b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:00.048725 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbbc437-c33b-4127-8be9-d6d4cbe480b4 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:00.049008 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbbc437-c33b-4127-8be9-d6d4cbe480b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:00.049397 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbbc437-c33b-4127-8be9-d6d4cbe480b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:00.076212 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:00.097315 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb7fe4f-93df-4101-a0a9-3df012beb621 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:00.123606 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbbc437-c33b-4127-8be9-d6d4cbe480b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:00.125064 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbbc437-c33b-4127-8be9-d6d4cbe480b4 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:00.133250 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a4e40-61b6-413b-af95-9c34caa1e9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:00.133937 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a4e40-61b6-413b-af95-9c34caa1e9ff None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:00.134239 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a4e40-61b6-413b-af95-9c34caa1e9ff None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:00.134494 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a4e40-61b6-413b-af95-9c34caa1e9ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:00.135514 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578a4e40-61b6-413b-af95-9c34caa1e9ff None admin] RBAC: Adding query filter params (role.id=6ca1ae4292d9414ca6526c6e8608de9e, scope.project.id=180b0af5863349e99ae412b07c1e9465, user.id=d15c5f6aeb874191b73c8ee0410c6bc1) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:00.135824 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578a4e40-61b6-413b-af95-9c34caa1e9ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:00.137257 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578a4e40-61b6-413b-af95-9c34caa1e9ff None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:00.151248 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 328/666] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Jun 24 08:27:59 2022] GET /identity/v3/role_assignments?user.id=d15c5f6aeb874191b73c8ee0410c6bc1&scope.project.id=180b0af5863349e99ae412b07c1e9465&role.id=6ca1ae4292d9414ca6526c6e8608de9e => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:00.151248 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a4e40-61b6-413b-af95-9c34caa1e9ff None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:00.266193 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:00.274494 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 339/667] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:27:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:28:00.277280 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4534e1-8c71-4df0-b5c4-d2539f23c074 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:00.277651 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4534e1-8c71-4df0-b5c4-d2539f23c074 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:00.278050 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4534e1-8c71-4df0-b5c4-d2539f23c074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:00.285735 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a4e40-61b6-413b-af95-9c34caa1e9ff None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:00.303628 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-bd4534e1-8c71-4df0-b5c4-d2539f23c074 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:00.306657 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-bd4534e1-8c71-4df0-b5c4-d2539f23c074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:00.336179 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b95576-a86f-46f2-9951-f41a17611b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:00.337969 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b95576-a86f-46f2-9951-f41a17611b0e None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:00.338456 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b95576-a86f-46f2-9951-f41a17611b0e None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:00.338923 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b95576-a86f-46f2-9951-f41a17611b0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:00.355645 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b95576-a86f-46f2-9951-f41a17611b0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:00.356864 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b95576-a86f-46f2-9951-f41a17611b0e None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:00.384667 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-f1b95576-a86f-46f2-9951-f41a17611b0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:00.384667 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:00.384667 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:00.384667 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:00.384667 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:00.384667 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:00.384667 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:00.384667 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:00.384667 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:00.384667 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:00.384667 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:00.384667 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:00.384667 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:28:00.384667 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:28:00.384667 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:28:00.385600 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:28:00.385600 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:00.385600 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:00.385600 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:00.385600 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:00.385600 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:00.385600 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:00.385600 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:00.385600 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:00.385600 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:00.385600 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:00.385600 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:00.385600 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:00.385600 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:00.385600 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:00.385600 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:00.385600 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:00.386517 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:28:00.386517 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:28:00.386517 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:00.386517 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:00.386517 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 329/668] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:00.389870 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b95576-a86f-46f2-9951-f41a17611b0e None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:00.403838 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:00.428292 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 340/669] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:00.429609 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4534e1-8c71-4df0-b5c4-d2539f23c074 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:00.520895 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b95576-a86f-46f2-9951-f41a17611b0e None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:00.562008 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a2ae3f-306f-469a-86d3-b6f789f9ad5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:00.563054 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a2ae3f-306f-469a-86d3-b6f789f9ad5b None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:00.563336 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a2ae3f-306f-469a-86d3-b6f789f9ad5b None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:00.563609 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a2ae3f-306f-469a-86d3-b6f789f9ad5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:00.564150 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4534e1-8c71-4df0-b5c4-d2539f23c074 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:00.576276 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a2ae3f-306f-469a-86d3-b6f789f9ad5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:00.577793 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a2ae3f-306f-469a-86d3-b6f789f9ad5b None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:00.580792 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:00.598475 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-44a2ae3f-306f-469a-86d3-b6f789f9ad5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:00.598475 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:00.598475 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:00.598475 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:00.598475 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:00.598475 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:00.598475 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:00.598475 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:00.598475 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:00.598475 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:00.598475 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:00.598475 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:00.598475 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:28:00.598475 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:28:00.598475 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:28:00.599983 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:28:00.599983 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:00.599983 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:00.599983 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:00.599983 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:00.599983 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:00.599983 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:00.599983 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:00.599983 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:00.599983 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:00.599983 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:00.599983 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:00.599983 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:00.599983 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:00.599983 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:00.599983 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:00.599983 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:00.601028 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:28:00.601028 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:28:00.601028 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:00.601028 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:00.601028 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 330/670] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:00.601941 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a2ae3f-306f-469a-86d3-b6f789f9ad5b None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:00.609929 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f54b4b5-d2f5-4bba-9f9f-cbbd1c450bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:00.610597 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54b4b5-d2f5-4bba-9f9f-cbbd1c450bd8 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:00.610877 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54b4b5-d2f5-4bba-9f9f-cbbd1c450bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:00.611150 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54b4b5-d2f5-4bba-9f9f-cbbd1c450bd8 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:00.611878 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f54b4b5-d2f5-4bba-9f9f-cbbd1c450bd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:00.612180 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f54b4b5-d2f5-4bba-9f9f-cbbd1c450bd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:00.613572 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f54b4b5-d2f5-4bba-9f9f-cbbd1c450bd8 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:00.629757 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 341/671] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:28:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:00.631440 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51cc26b-663e-4e6c-a8d3-4c33e966086c None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:00.631937 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51cc26b-663e-4e6c-a8d3-4c33e966086c None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:00.632450 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51cc26b-663e-4e6c-a8d3-4c33e966086c None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:00.635060 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 342/672] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:00.636268 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51cc26b-663e-4e6c-a8d3-4c33e966086c None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:00.700104 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:00.732702 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a2ae3f-306f-469a-86d3-b6f789f9ad5b None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:00.762062 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51cc26b-663e-4e6c-a8d3-4c33e966086c None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:00.771116 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7614ef7-548f-4835-9e91-5a4763ccac28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:00.772164 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7614ef7-548f-4835-9e91-5a4763ccac28 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:00.772449 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7614ef7-548f-4835-9e91-5a4763ccac28 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:00.772723 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7614ef7-548f-4835-9e91-5a4763ccac28 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:00.783850 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7614ef7-548f-4835-9e91-5a4763ccac28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:00.791679 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-a7614ef7-548f-4835-9e91-5a4763ccac28 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:28:00.791679 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:00.791679 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:00.791679 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:00.791679 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:00.791679 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:00.791679 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:00.791679 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:00.791679 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:00.791679 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:00.791679 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:00.791679 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:00.791679 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:00.791679 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:00.791679 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:00.791679 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:00.792807 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:00.792807 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:00.792807 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:00.792807 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:00.792807 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:00.792807 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:00.792807 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:00.792807 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:00.792807 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:00.792807 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:00.792807 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:00.792807 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:00.792807 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:00.792807 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:00.792807 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:00.792807 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:00.795263 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:00.795263 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:00.795263 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:00.795263 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:00.795263 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:00.795263 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:00.795263 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:00.795263 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:00.795263 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:00.795263 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:00.795263 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:28:00.795263 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:00.795263 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 331/673] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Jun 24 08:28:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:00.795263 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7614ef7-548f-4835-9e91-5a4763ccac28 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:00.806068 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348a3fbc-458d-4543-a4ea-2e0c712d84f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:00.807098 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a3fbc-458d-4543-a4ea-2e0c712d84f4 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:00.807098 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a3fbc-458d-4543-a4ea-2e0c712d84f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:00.807385 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a3fbc-458d-4543-a4ea-2e0c712d84f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:00.807955 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348a3fbc-458d-4543-a4ea-2e0c712d84f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:00.808253 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348a3fbc-458d-4543-a4ea-2e0c712d84f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:00.809663 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348a3fbc-458d-4543-a4ea-2e0c712d84f4 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:00.822932 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 343/674] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:28:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:00.823997 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5ddc8-860b-4086-94dd-eb6e64def57c None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:00.824279 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5ddc8-860b-4086-94dd-eb6e64def57c None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:00.824556 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5ddc8-860b-4086-94dd-eb6e64def57c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:00.841318 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-16c5ddc8-860b-4086-94dd-eb6e64def57c None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:00.842544 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-16c5ddc8-860b-4086-94dd-eb6e64def57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:00.861916 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:00.911431 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7614ef7-548f-4835-9e91-5a4763ccac28 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:00.921730 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 344/675] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:00.921730 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5ddc8-860b-4086-94dd-eb6e64def57c None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:00.959792 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:00.966343 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155d71e2-f6d5-41a1-9bf8-2954da18bc01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:00.967155 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155d71e2-f6d5-41a1-9bf8-2954da18bc01 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:00.967423 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155d71e2-f6d5-41a1-9bf8-2954da18bc01 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:00.968128 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155d71e2-f6d5-41a1-9bf8-2954da18bc01 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:00.975721 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155d71e2-f6d5-41a1-9bf8-2954da18bc01 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:00.977367 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155d71e2-f6d5-41a1-9bf8-2954da18bc01 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:00.991472 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-155d71e2-f6d5-41a1-9bf8-2954da18bc01 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:28:00.991472 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:00.991472 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:00.991472 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:00.991472 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:00.991472 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:00.991472 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:00.991472 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:00.991472 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:00.991472 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:00.991472 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:00.991472 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:00.991472 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:28:00.991472 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:28:00.991472 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:28:00.992415 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:28:00.992415 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:00.992415 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:00.992415 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:00.992415 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:00.992415 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:00.992415 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:00.992415 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:00.992415 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:00.992415 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:00.992415 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:00.992415 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:00.992415 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:00.992415 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:00.992415 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:00.992415 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:00.992415 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:00.994102 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:28:00.994102 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:28:00.994102 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:28:00.994102 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:28:00.994102 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:28:00.994102 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:28:00.994102 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:28:00.994102 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:00.994102 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 332/676] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:28:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:00.994102 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155d71e2-f6d5-41a1-9bf8-2954da18bc01 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:01.041476 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5ddc8-860b-4086-94dd-eb6e64def57c None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:01.080287 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16ed3bf-7912-47d8-a15b-46485c686192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:01.080954 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ed3bf-7912-47d8-a15b-46485c686192 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:01.081238 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ed3bf-7912-47d8-a15b-46485c686192 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:01.081513 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ed3bf-7912-47d8-a15b-46485c686192 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:01.082400 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16ed3bf-7912-47d8-a15b-46485c686192 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:01.082764 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16ed3bf-7912-47d8-a15b-46485c686192 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:01.084304 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16ed3bf-7912-47d8-a15b-46485c686192 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:01.096183 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 345/677] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:28:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:01.101009 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155d71e2-f6d5-41a1-9bf8-2954da18bc01 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:01.101559 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16ed3bf-7912-47d8-a15b-46485c686192 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:01.119023 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:01.138657 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579db664-e5ea-4e2a-81ec-d47c9ec8501a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:01.139448 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579db664-e5ea-4e2a-81ec-d47c9ec8501a None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:01.139800 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579db664-e5ea-4e2a-81ec-d47c9ec8501a None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:01.140014 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579db664-e5ea-4e2a-81ec-d47c9ec8501a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:01.148399 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579db664-e5ea-4e2a-81ec-d47c9ec8501a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:01.149983 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579db664-e5ea-4e2a-81ec-d47c9ec8501a None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:01.163621 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-579db664-e5ea-4e2a-81ec-d47c9ec8501a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:28:01.163621 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:01.163621 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:01.163621 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:01.163621 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:01.163621 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:01.163621 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:01.163621 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:01.163621 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:01.163621 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:01.163621 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:01.163621 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:01.163621 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:28:01.163621 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:28:01.163621 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:28:01.165079 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:28:01.165079 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:01.165079 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:01.165079 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:01.165079 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:01.165079 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:01.165079 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:01.165079 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:01.165079 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:01.165079 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:01.165079 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:01.165079 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:01.165079 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:01.165079 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:01.165079 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:01.165079 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:01.165079 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:01.166151 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:28:01.166151 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:28:01.166151 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:28:01.166151 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:28:01.166151 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:28:01.166151 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:28:01.166151 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:28:01.166151 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:01.166151 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 333/678] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:28:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:01.166654 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579db664-e5ea-4e2a-81ec-d47c9ec8501a None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:01.218334 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:01.226278 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16ed3bf-7912-47d8-a15b-46485c686192 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:01.264483 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585b6c43-54ec-486e-b7cc-b4f174b210c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:01.265273 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585b6c43-54ec-486e-b7cc-b4f174b210c4 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:01.265555 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585b6c43-54ec-486e-b7cc-b4f174b210c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:01.265940 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585b6c43-54ec-486e-b7cc-b4f174b210c4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:01.273650 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579db664-e5ea-4e2a-81ec-d47c9ec8501a None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:01.276805 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585b6c43-54ec-486e-b7cc-b4f174b210c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:01.284414 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-585b6c43-54ec-486e-b7cc-b4f174b210c4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:28:01.284414 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:01.284414 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:01.284414 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:01.284414 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:01.284414 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:01.284414 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:01.284414 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:01.284414 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:01.284414 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:01.284414 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:01.284414 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:01.284414 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:01.284414 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:01.284414 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:01.284414 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:01.285342 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:01.285342 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:01.285342 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:01.285342 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:01.285342 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:01.285342 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:01.285342 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:01.285342 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:01.285342 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:01.285342 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:01.285342 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:01.285342 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:01.285342 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:01.285342 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:01.285342 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:01.285342 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:01.286741 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:01.286741 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:01.286741 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:01.286741 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:01.286741 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:01.286741 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:01.286741 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:01.286741 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:01.286741 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:01.286741 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:01.286741 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:28:01.286741 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:01.286741 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 346/679] 10.222.0.13 () {56 vars in 1102 bytes} [Fri Jun 24 08:28:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:01.286741 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585b6c43-54ec-486e-b7cc-b4f174b210c4 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:01.315992 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c83aeb-260e-4d40-b351-5849db9633b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:01.316574 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c83aeb-260e-4d40-b351-5849db9633b5 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:01.316855 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c83aeb-260e-4d40-b351-5849db9633b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:01.317129 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c83aeb-260e-4d40-b351-5849db9633b5 None admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:01.317930 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c83aeb-260e-4d40-b351-5849db9633b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:01.318238 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c83aeb-260e-4d40-b351-5849db9633b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:01.319666 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c83aeb-260e-4d40-b351-5849db9633b5 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:01.336364 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 334/680] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Jun 24 08:28:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:01.336364 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c83aeb-260e-4d40-b351-5849db9633b5 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:01.379248 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:01.404315 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585b6c43-54ec-486e-b7cc-b4f174b210c4 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:01.444248 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8cf048-7399-4bd7-8979-b34abc8eab17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:01.445174 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8cf048-7399-4bd7-8979-b34abc8eab17 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:01.445486 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8cf048-7399-4bd7-8979-b34abc8eab17 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:01.446631 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8cf048-7399-4bd7-8979-b34abc8eab17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:01.449652 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c83aeb-260e-4d40-b351-5849db9633b5 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:01.458606 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8cf048-7399-4bd7-8979-b34abc8eab17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:01.461037 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8cf048-7399-4bd7-8979-b34abc8eab17 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:01.479523 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-2d8cf048-7399-4bd7-8979-b34abc8eab17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:01.479523 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:01.479523 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:01.479523 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:01.479523 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:01.479523 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:01.479523 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:01.479523 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:01.479523 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:01.479523 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:01.479523 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:01.479523 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:01.479523 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:28:01.479523 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:28:01.479523 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:28:01.480526 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:28:01.480526 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:01.480526 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:01.480526 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:01.480526 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:01.480526 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:01.480526 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:01.480526 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:01.480526 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:01.480526 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:01.480526 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:01.480526 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:01.480526 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:01.480526 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:01.480526 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:01.480526 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:01.480526 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:01.481398 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:28:01.481398 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:28:01.481398 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:01.481398 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:01.481398 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 347/681] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:01.482542 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8cf048-7399-4bd7-8979-b34abc8eab17 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:01.491049 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8186f1a8-f5ff-49a9-b9e7-aa04f97e84da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:01.491718 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8186f1a8-f5ff-49a9-b9e7-aa04f97e84da None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:01.491998 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8186f1a8-f5ff-49a9-b9e7-aa04f97e84da None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:01.492277 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8186f1a8-f5ff-49a9-b9e7-aa04f97e84da None admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:01.493582 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8186f1a8-f5ff-49a9-b9e7-aa04f97e84da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:01.493926 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8186f1a8-f5ff-49a9-b9e7-aa04f97e84da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:01.495360 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8186f1a8-f5ff-49a9-b9e7-aa04f97e84da None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:01.496277 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:01.513027 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 335/682] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Jun 24 08:28:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:01.513027 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8186f1a8-f5ff-49a9-b9e7-aa04f97e84da None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:01.608729 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8cf048-7399-4bd7-8979-b34abc8eab17 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:01.611225 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:01.619198 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8186f1a8-f5ff-49a9-b9e7-aa04f97e84da None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:01.644724 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f2815f-ffd0-44bd-b2d4-2ed8ad04d5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:01.645427 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2815f-ffd0-44bd-b2d4-2ed8ad04d5bc None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:01.645774 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2815f-ffd0-44bd-b2d4-2ed8ad04d5bc None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:01.646032 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2815f-ffd0-44bd-b2d4-2ed8ad04d5bc None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:01.646813 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f2815f-ffd0-44bd-b2d4-2ed8ad04d5bc None admin] RBAC: Adding query filter params (name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:01.647120 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f2815f-ffd0-44bd-b2d4-2ed8ad04d5bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:01.648507 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f2815f-ffd0-44bd-b2d4-2ed8ad04d5bc None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:01.654639 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d4865-7614-4172-8bd0-73d50e61c182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:01.655360 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d4865-7614-4172-8bd0-73d50e61c182 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:01.655646 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d4865-7614-4172-8bd0-73d50e61c182 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:01.655917 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d4865-7614-4172-8bd0-73d50e61c182 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:01.656735 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d4865-7614-4172-8bd0-73d50e61c182 None admin] RBAC: Adding query filter params (role.id=6ca1ae4292d9414ca6526c6e8608de9e, scope.project.id=180b0af5863349e99ae412b07c1e9465, user.id=44dcdfb5ca1541f9b4967a1f8ddbb34b) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:01.657045 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d4865-7614-4172-8bd0-73d50e61c182 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:01.658493 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d4865-7614-4172-8bd0-73d50e61c182 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:01.659374 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 348/683] 10.222.0.13 () {56 vars in 1103 bytes} [Fri Jun 24 08:28:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:01.660371 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f2815f-ffd0-44bd-b2d4-2ed8ad04d5bc None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:01.669041 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 336/684] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Jun 24 08:28:01 2022] GET /identity/v3/role_assignments?user.id=44dcdfb5ca1541f9b4967a1f8ddbb34b&scope.project.id=180b0af5863349e99ae412b07c1e9465&role.id=6ca1ae4292d9414ca6526c6e8608de9e => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:01.670052 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d4865-7614-4172-8bd0-73d50e61c182 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:01.777139 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:01.800793 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d4865-7614-4172-8bd0-73d50e61c182 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:01.826664 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f2815f-ffd0-44bd-b2d4-2ed8ad04d5bc None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:01.841230 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53166b80-47d8-4de9-8c28-352b56a09992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:01.842266 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53166b80-47d8-4de9-8c28-352b56a09992 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:01.842752 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53166b80-47d8-4de9-8c28-352b56a09992 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:01.843196 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53166b80-47d8-4de9-8c28-352b56a09992 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:01.844027 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53166b80-47d8-4de9-8c28-352b56a09992 None admin] RBAC: Adding query filter params (role.id=6ca1ae4292d9414ca6526c6e8608de9e, scope.project.id=180b0af5863349e99ae412b07c1e9465, user.id=c5f7ba6c7faf43e7acbb43eee58d8c8d) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:01.844351 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53166b80-47d8-4de9-8c28-352b56a09992 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:01.846609 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53166b80-47d8-4de9-8c28-352b56a09992 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:01.853769 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 337/685] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Jun 24 08:28:01 2022] GET /identity/v3/role_assignments?user.id=c5f7ba6c7faf43e7acbb43eee58d8c8d&scope.project.id=180b0af5863349e99ae412b07c1e9465&role.id=6ca1ae4292d9414ca6526c6e8608de9e => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:01.854103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53166b80-47d8-4de9-8c28-352b56a09992 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:01.870206 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19d5d9-ff99-4456-97ce-3a1cf141367f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:01.871370 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19d5d9-ff99-4456-97ce-3a1cf141367f None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:01.871995 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19d5d9-ff99-4456-97ce-3a1cf141367f None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:01.872344 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19d5d9-ff99-4456-97ce-3a1cf141367f None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:01.873167 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a19d5d9-ff99-4456-97ce-3a1cf141367f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:01.873593 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a19d5d9-ff99-4456-97ce-3a1cf141367f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:01.875230 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a19d5d9-ff99-4456-97ce-3a1cf141367f None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:01.892770 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:01.897214 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 349/686] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:28:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:01.906817 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19d5d9-ff99-4456-97ce-3a1cf141367f None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:01.991240 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53166b80-47d8-4de9-8c28-352b56a09992 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:02.029992 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19d5d9-ff99-4456-97ce-3a1cf141367f None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:02.037138 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74ef8f6-e968-4fa0-922d-612d03a684e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:02.038138 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ef8f6-e968-4fa0-922d-612d03a684e9 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:02.038525 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ef8f6-e968-4fa0-922d-612d03a684e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:02.038902 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ef8f6-e968-4fa0-922d-612d03a684e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:02.056043 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74ef8f6-e968-4fa0-922d-612d03a684e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:02.058156 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74ef8f6-e968-4fa0-922d-612d03a684e9 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:02.065762 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:02.082352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-e74ef8f6-e968-4fa0-922d-612d03a684e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:02.082352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:02.082352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:02.082352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:02.082352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:02.082352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:02.082352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:02.082352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:02.082352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:02.082352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:02.082352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:02.082352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:02.082352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:28:02.082352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:28:02.082352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:28:02.083391 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:28:02.083391 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:02.083391 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:02.083391 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:02.083391 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:02.083391 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:02.083391 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:02.083391 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:02.083391 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:02.083391 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:02.083391 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:02.083391 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:02.083391 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:02.083391 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:02.083391 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:02.083391 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:02.083391 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:02.084318 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:28:02.084318 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:28:02.084318 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:02.084318 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:02.087384 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 338/687] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:02.087674 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60df01a8-64e3-4ee0-bc0e-b7b65b645cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:02.088476 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df01a8-64e3-4ee0-bc0e-b7b65b645cb7 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:02.088756 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df01a8-64e3-4ee0-bc0e-b7b65b645cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:02.089030 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df01a8-64e3-4ee0-bc0e-b7b65b645cb7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:02.092702 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74ef8f6-e968-4fa0-922d-612d03a684e9 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:02.109722 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60df01a8-64e3-4ee0-bc0e-b7b65b645cb7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:02.109722 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60df01a8-64e3-4ee0-bc0e-b7b65b645cb7 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:02.124268 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-60df01a8-64e3-4ee0-bc0e-b7b65b645cb7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 08:28:02.124268 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:02.124268 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:02.124268 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:02.124268 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:02.124268 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:02.124268 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:02.124268 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:02.124268 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:02.124268 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:02.124268 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:02.124268 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:02.124268 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:28:02.124268 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:28:02.124268 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:28:02.124268 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:28:02.125222 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:02.125222 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:02.125222 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:28:02.125222 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:28:02.125222 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:28:02.125222 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:28:02.125222 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:02.125222 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:02.125222 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:02.125222 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:02.125222 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:02.125222 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:02.125222 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:02.125222 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:02.125222 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:02.125222 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:02.125222 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:02.125222 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:02.126458 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:02.126458 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:02.126458 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:28:02.126458 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:28:02.126458 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:28:02.126458 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:28:02.126458 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:28:02.126458 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:28:02.126458 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 08:28:02.126458 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:02.126458 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 350/688] 10.222.0.13 () {56 vars in 1093 bytes} [Fri Jun 24 08:28:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:02.134043 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60df01a8-64e3-4ee0-bc0e-b7b65b645cb7 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:02.212988 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:02.245549 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74ef8f6-e968-4fa0-922d-612d03a684e9 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:02.271371 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60df01a8-64e3-4ee0-bc0e-b7b65b645cb7 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:02.289981 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8b714b-e5fd-4855-a873-3ddade472dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:02.290842 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b714b-e5fd-4855-a873-3ddade472dc6 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:02.292075 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b714b-e5fd-4855-a873-3ddade472dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:02.292352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b714b-e5fd-4855-a873-3ddade472dc6 None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:02.293079 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8b714b-e5fd-4855-a873-3ddade472dc6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:02.293383 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8b714b-e5fd-4855-a873-3ddade472dc6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:02.294826 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8b714b-e5fd-4855-a873-3ddade472dc6 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:02.312823 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 339/689] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:28:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:02.314301 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8b714b-e5fd-4855-a873-3ddade472dc6 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:02.317135 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b420a-9b83-49fd-9e68-34a373dc0697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:02.319215 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b420a-9b83-49fd-9e68-34a373dc0697 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:02.319712 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b420a-9b83-49fd-9e68-34a373dc0697 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:02.320198 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b420a-9b83-49fd-9e68-34a373dc0697 None admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:02.321814 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360b420a-9b83-49fd-9e68-34a373dc0697 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:02.322345 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360b420a-9b83-49fd-9e68-34a373dc0697 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:02.324006 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360b420a-9b83-49fd-9e68-34a373dc0697 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:02.350165 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 351/690] 10.222.0.13 () {56 vars in 1133 bytes} [Fri Jun 24 08:28:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:02.356066 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b420a-9b83-49fd-9e68-34a373dc0697 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:02.427808 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:02.435464 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8b714b-e5fd-4855-a873-3ddade472dc6 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:02.475268 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e6ffbf-e0fc-4de6-a6cf-72a80f8b62f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:02.476091 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e6ffbf-e0fc-4de6-a6cf-72a80f8b62f4 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:02.476371 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e6ffbf-e0fc-4de6-a6cf-72a80f8b62f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:02.476644 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e6ffbf-e0fc-4de6-a6cf-72a80f8b62f4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:02.480705 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:02.487987 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e6ffbf-e0fc-4de6-a6cf-72a80f8b62f4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:02.490139 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e6ffbf-e0fc-4de6-a6cf-72a80f8b62f4 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:02.500535 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b420a-9b83-49fd-9e68-34a373dc0697 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:02.506822 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-32e6ffbf-e0fc-4de6-a6cf-72a80f8b62f4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 08:28:02.506822 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:02.506822 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:02.506822 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:02.506822 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:02.506822 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:02.506822 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:02.506822 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:02.506822 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:02.506822 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:02.506822 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:02.506822 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:02.506822 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:28:02.506822 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:28:02.506822 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:28:02.506822 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:28:02.508050 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:02.508050 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:02.508050 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:28:02.508050 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:28:02.508050 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:28:02.508050 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:28:02.508050 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:02.508050 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:02.508050 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:02.508050 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:02.508050 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:02.508050 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:02.508050 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:02.508050 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:02.508050 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:02.508050 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:02.508050 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:02.508050 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:02.509570 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:02.509570 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:02.509570 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:28:02.509570 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:28:02.509570 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:28:02.509570 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:28:02.509570 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:28:02.509570 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:28:02.509570 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 08:28:02.509570 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:02.515566 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 340/691] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Jun 24 08:28:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:02.517196 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593cc0e2-47a4-452a-bbbc-7cbb7cb72be5 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:02.517196 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593cc0e2-47a4-452a-bbbc-7cbb7cb72be5 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:02.517653 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593cc0e2-47a4-452a-bbbc-7cbb7cb72be5 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:02.519787 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 341/692] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:02.520688 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593cc0e2-47a4-452a-bbbc-7cbb7cb72be5 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:02.546654 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0736906-701b-4b79-b640-df6217e54ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:02.547502 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0736906-701b-4b79-b640-df6217e54ab5 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:02.547899 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0736906-701b-4b79-b640-df6217e54ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:02.548267 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0736906-701b-4b79-b640-df6217e54ab5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:02.622635 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0736906-701b-4b79-b640-df6217e54ab5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:02.624250 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0736906-701b-4b79-b640-df6217e54ab5 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:02.660577 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593cc0e2-47a4-452a-bbbc-7cbb7cb72be5 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:02.679101 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:02.713724 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cae48e-f7cb-47a2-aa55-9cd12e5678dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:02.715089 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cae48e-f7cb-47a2-aa55-9cd12e5678dc None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:02.715530 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cae48e-f7cb-47a2-aa55-9cd12e5678dc None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:02.716205 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cae48e-f7cb-47a2-aa55-9cd12e5678dc None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:02.717191 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cae48e-f7cb-47a2-aa55-9cd12e5678dc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:02.717909 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cae48e-f7cb-47a2-aa55-9cd12e5678dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:02.720242 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cae48e-f7cb-47a2-aa55-9cd12e5678dc None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:02.749300 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 342/693] 10.222.0.13 () {56 vars in 1135 bytes} [Fri Jun 24 08:28:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:02.749300 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb3736-ba08-48ba-a354-bf7c4695e39c None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:02.749300 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb3736-ba08-48ba-a354-bf7c4695e39c None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:02.749300 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb3736-ba08-48ba-a354-bf7c4695e39c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:02.750017 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 352/694] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:28:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 08:28:02.751628 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0736906-701b-4b79-b640-df6217e54ab5 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:02.766538 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-e7fb3736-ba08-48ba-a354-bf7c4695e39c None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:02.767814 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-e7fb3736-ba08-48ba-a354-bf7c4695e39c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:02.819608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:02.838833 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 343/695] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:02.839836 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb3736-ba08-48ba-a354-bf7c4695e39c None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:02.886980 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0736906-701b-4b79-b640-df6217e54ab5 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:02.924037 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e2fcb9-0415-49b9-a0e5-59e398dc77ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:02.924924 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e2fcb9-0415-49b9-a0e5-59e398dc77ac None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:02.925276 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e2fcb9-0415-49b9-a0e5-59e398dc77ac None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:02.925635 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e2fcb9-0415-49b9-a0e5-59e398dc77ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:02.938269 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e2fcb9-0415-49b9-a0e5-59e398dc77ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:02.939512 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e2fcb9-0415-49b9-a0e5-59e398dc77ac None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:02.949067 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:02.959081 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-00e2fcb9-0415-49b9-a0e5-59e398dc77ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:02.959081 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:02.959081 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:02.959081 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:02.959081 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:02.959081 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:02.959081 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:02.959081 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:02.959081 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:02.959081 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:02.959081 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:02.959081 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:02.959081 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:28:02.959081 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:28:02.959081 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:28:02.959991 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:28:02.959991 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:02.959991 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:02.959991 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:02.959991 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:02.959991 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:02.959991 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:02.959991 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:02.959991 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:02.959991 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:02.959991 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:02.959991 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:02.959991 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:02.959991 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:02.959991 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:02.959991 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:02.959991 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:02.960855 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:28:02.960855 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:28:02.960855 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:02.960855 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:02.962930 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb3736-ba08-48ba-a354-bf7c4695e39c None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:02.966215 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 353/696] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:02.966695 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e2fcb9-0415-49b9-a0e5-59e398dc77ac None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:03.007465 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f678b46-0e08-417b-99fa-81c393fd66a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:03.008233 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f678b46-0e08-417b-99fa-81c393fd66a8 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:03.008596 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f678b46-0e08-417b-99fa-81c393fd66a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:03.008941 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f678b46-0e08-417b-99fa-81c393fd66a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:03.063390 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:03.081656 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f678b46-0e08-417b-99fa-81c393fd66a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:03.083402 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f678b46-0e08-417b-99fa-81c393fd66a8 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:03.084170 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e2fcb9-0415-49b9-a0e5-59e398dc77ac None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:03.122032 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad739af-6828-4053-a8a4-61033d84376d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:03.123158 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad739af-6828-4053-a8a4-61033d84376d None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:03.123663 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad739af-6828-4053-a8a4-61033d84376d None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:03.124131 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad739af-6828-4053-a8a4-61033d84376d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:03.142240 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad739af-6828-4053-a8a4-61033d84376d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:03.143771 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad739af-6828-4053-a8a4-61033d84376d None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:03.170252 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-fad739af-6828-4053-a8a4-61033d84376d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:03.170252 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:03.170252 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:03.170252 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:03.170252 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:03.170252 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:03.170252 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:03.170252 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:03.170252 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:03.170252 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:03.170252 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:03.170252 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:03.170252 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:28:03.170252 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:28:03.170252 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:28:03.171209 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:28:03.171209 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:03.171209 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:03.171209 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:03.171209 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:03.171209 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:03.171209 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:03.171209 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:03.171209 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:03.171209 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:03.171209 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:03.171209 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:03.171209 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:03.171209 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:03.171209 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:03.171209 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:03.171209 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:03.172088 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:28:03.172088 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:28:03.172088 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:03.172088 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:03.176673 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 354/697] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:03.177517 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad739af-6828-4053-a8a4-61033d84376d None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:03.208932 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 344/698] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:28:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:28:03.209747 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f678b46-0e08-417b-99fa-81c393fd66a8 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:03.238621 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:03.292140 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad739af-6828-4053-a8a4-61033d84376d None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:03.322137 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:03.340555 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f678b46-0e08-417b-99fa-81c393fd66a8 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:03.341524 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333d890-202e-474f-85f2-fe528fab73da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:03.342231 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333d890-202e-474f-85f2-fe528fab73da None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:03.342519 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333d890-202e-474f-85f2-fe528fab73da None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:03.342799 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333d890-202e-474f-85f2-fe528fab73da None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:03.343834 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7333d890-202e-474f-85f2-fe528fab73da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:03.344150 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7333d890-202e-474f-85f2-fe528fab73da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:03.346394 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7333d890-202e-474f-85f2-fe528fab73da None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:03.361861 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 355/699] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:28:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:03.362948 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333d890-202e-474f-85f2-fe528fab73da None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:03.382963 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f7a1d0-5905-4cc4-a73c-0efc645ceb7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:03.384545 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f7a1d0-5905-4cc4-a73c-0efc645ceb7f None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:03.385049 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f7a1d0-5905-4cc4-a73c-0efc645ceb7f None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:03.385514 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f7a1d0-5905-4cc4-a73c-0efc645ceb7f None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:03.386463 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f7a1d0-5905-4cc4-a73c-0efc645ceb7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:03.386963 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f7a1d0-5905-4cc4-a73c-0efc645ceb7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:03.388545 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f7a1d0-5905-4cc4-a73c-0efc645ceb7f None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:03.401326 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 345/700] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:28:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:03.402484 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f7a1d0-5905-4cc4-a73c-0efc645ceb7f None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:03.483500 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333d890-202e-474f-85f2-fe528fab73da None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:03.499956 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:03.523545 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be96468-4395-4d80-94f8-aeaf7b3648f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:03.525262 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be96468-4395-4d80-94f8-aeaf7b3648f9 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:03.525791 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be96468-4395-4d80-94f8-aeaf7b3648f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:03.526261 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be96468-4395-4d80-94f8-aeaf7b3648f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:03.537849 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f7a1d0-5905-4cc4-a73c-0efc645ceb7f None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:03.538338 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be96468-4395-4d80-94f8-aeaf7b3648f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:03.540318 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be96468-4395-4d80-94f8-aeaf7b3648f9 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:03.558787 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-8be96468-4395-4d80-94f8-aeaf7b3648f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:03.558787 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:03.558787 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:03.558787 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:03.558787 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:03.558787 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:03.558787 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:03.558787 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:03.558787 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:03.558787 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:03.558787 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:03.558787 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:03.558787 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:28:03.558787 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:28:03.558787 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:28:03.559690 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:28:03.559690 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:03.559690 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:03.559690 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:03.559690 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:03.559690 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:03.559690 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:03.559690 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:03.559690 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:03.559690 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:03.559690 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:03.559690 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:03.559690 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:03.559690 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:03.559690 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:03.559690 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:03.559690 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:03.560637 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:28:03.560637 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:28:03.560637 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:03.560637 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:03.562931 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 356/701] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:03.564072 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be96468-4395-4d80-94f8-aeaf7b3648f9 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:03.580146 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b16b87-54a2-4e56-be6a-e1d9211b337a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:03.580952 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b16b87-54a2-4e56-be6a-e1d9211b337a None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:03.581242 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b16b87-54a2-4e56-be6a-e1d9211b337a None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:03.581869 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b16b87-54a2-4e56-be6a-e1d9211b337a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:03.592858 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b16b87-54a2-4e56-be6a-e1d9211b337a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:03.594627 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b16b87-54a2-4e56-be6a-e1d9211b337a None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:03.611395 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-04b16b87-54a2-4e56-be6a-e1d9211b337a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:28:03.611395 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:03.611395 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:03.611395 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:03.611395 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:03.611395 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:03.611395 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:03.611395 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:03.611395 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:03.611395 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:03.611395 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:03.611395 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:03.611395 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:28:03.611395 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:28:03.611395 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:28:03.612297 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:28:03.612297 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:03.612297 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:03.612297 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:03.612297 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:03.612297 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:03.612297 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:03.612297 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:03.612297 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:03.612297 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:03.612297 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:03.612297 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:03.612297 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:03.612297 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:03.612297 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:03.612297 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:03.612297 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:03.613242 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:28:03.613242 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:28:03.613242 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:28:03.613242 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:28:03.613242 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:28:03.613242 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:28:03.613242 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:28:03.613242 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:03.615789 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 346/702] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:28:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:03.616791 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b16b87-54a2-4e56-be6a-e1d9211b337a None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:03.621233 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:03.688489 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be96468-4395-4d80-94f8-aeaf7b3648f9 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:03.727087 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed32601-a138-4b81-9407-1b9d9807c68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:03.728212 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed32601-a138-4b81-9407-1b9d9807c68f None admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:03.728692 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed32601-a138-4b81-9407-1b9d9807c68f None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:03.729157 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed32601-a138-4b81-9407-1b9d9807c68f None admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:03.730206 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed32601-a138-4b81-9407-1b9d9807c68f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:03.731883 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed32601-a138-4b81-9407-1b9d9807c68f None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:03.733236 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b16b87-54a2-4e56-be6a-e1d9211b337a None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:03.759003 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 357/703] 10.222.0.13 () {60 vars in 1132 bytes} [Fri Jun 24 08:28:03 2022] POST /identity/v3/users => generated 254 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:28:03.760394 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed32601-a138-4b81-9407-1b9d9807c68f None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:03.792259 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:03.801814 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cd1630-db5b-4015-a146-dcf0cff28913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:03.802428 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd1630-db5b-4015-a146-dcf0cff28913 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:03.802717 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd1630-db5b-4015-a146-dcf0cff28913 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:03.802997 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd1630-db5b-4015-a146-dcf0cff28913 None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:03.803731 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cd1630-db5b-4015-a146-dcf0cff28913 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:03.804033 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cd1630-db5b-4015-a146-dcf0cff28913 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:03.806136 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cd1630-db5b-4015-a146-dcf0cff28913 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:03.833667 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 347/704] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:28:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:03.842767 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cd1630-db5b-4015-a146-dcf0cff28913 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:03.898101 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed32601-a138-4b81-9407-1b9d9807c68f None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:03.942768 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062f34e2-3bbb-4b36-bcfa-2a5fbf545e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:03.944444 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f34e2-3bbb-4b36-bcfa-2a5fbf545e68 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:03.944934 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f34e2-3bbb-4b36-bcfa-2a5fbf545e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:03.945429 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f34e2-3bbb-4b36-bcfa-2a5fbf545e68 None admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:03.947036 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062f34e2-3bbb-4b36-bcfa-2a5fbf545e68 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:03.947555 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062f34e2-3bbb-4b36-bcfa-2a5fbf545e68 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:03.949813 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062f34e2-3bbb-4b36-bcfa-2a5fbf545e68 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:03.966003 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 358/705] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Jun 24 08:28:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:03.971538 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062f34e2-3bbb-4b36-bcfa-2a5fbf545e68 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:03.999230 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:04.061864 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cd1630-db5b-4015-a146-dcf0cff28913 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:04.080663 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:04.101220 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17063dc4-a8fd-4a1b-8d66-590519ebdfb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:04.102655 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17063dc4-a8fd-4a1b-8d66-590519ebdfb7 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:04.102960 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17063dc4-a8fd-4a1b-8d66-590519ebdfb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:04.103293 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17063dc4-a8fd-4a1b-8d66-590519ebdfb7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:04.107998 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062f34e2-3bbb-4b36-bcfa-2a5fbf545e68 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:04.111532 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17063dc4-a8fd-4a1b-8d66-590519ebdfb7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:04.112930 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17063dc4-a8fd-4a1b-8d66-590519ebdfb7 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:04.132367 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-17063dc4-a8fd-4a1b-8d66-590519ebdfb7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:28:04.132367 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:04.132367 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:04.132367 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:04.132367 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:04.132367 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:04.132367 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:04.132367 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:04.132367 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:04.132367 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:04.132367 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:04.132367 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:04.132367 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:28:04.132367 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:28:04.132367 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:28:04.133379 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:28:04.133379 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:04.133379 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:04.133379 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:04.133379 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:04.133379 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:04.133379 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:04.133379 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:04.133379 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:04.133379 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:04.133379 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:04.133379 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:04.133379 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:04.133379 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:04.133379 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:04.133379 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:04.133379 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:04.134324 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:28:04.134324 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:28:04.134324 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:28:04.134324 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:28:04.134324 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:28:04.134324 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:28:04.134324 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:28:04.134324 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:04.134324 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 348/706] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:28:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:28:04.139664 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17063dc4-a8fd-4a1b-8d66-590519ebdfb7 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:04.149750 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17806ebf-c9d4-4b64-bd4d-e79152c4a29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:04.150412 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17806ebf-c9d4-4b64-bd4d-e79152c4a29f None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:04.150701 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17806ebf-c9d4-4b64-bd4d-e79152c4a29f None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:04.150985 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17806ebf-c9d4-4b64-bd4d-e79152c4a29f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:04.152077 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17806ebf-c9d4-4b64-bd4d-e79152c4a29f None admin] RBAC: Adding query filter params (role.id=acfe707eee6944ad94d1915fc9ee7ea3, scope.project.id=180b0af5863349e99ae412b07c1e9465, user.id=74bb1bc6821b4500b26f160ccae5b7d5) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:04.152188 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17806ebf-c9d4-4b64-bd4d-e79152c4a29f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:04.153552 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17806ebf-c9d4-4b64-bd4d-e79152c4a29f None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:04.160757 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 359/707] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Jun 24 08:28:03 2022] GET /identity/v3/role_assignments?user.id=74bb1bc6821b4500b26f160ccae5b7d5&scope.project.id=180b0af5863349e99ae412b07c1e9465&role.id=acfe707eee6944ad94d1915fc9ee7ea3 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:04.173917 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaff136-f24b-4b0d-b9cf-689ad3aac25f None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:04.174243 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaff136-f24b-4b0d-b9cf-689ad3aac25f None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:04.174530 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaff136-f24b-4b0d-b9cf-689ad3aac25f None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:04.176022 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 360/708] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:28:04.182397 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd333e20-11ed-4eeb-a311-1ad0cbb1a785 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:04.182714 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd333e20-11ed-4eeb-a311-1ad0cbb1a785 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:04.182971 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd333e20-11ed-4eeb-a311-1ad0cbb1a785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:04.203726 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-bd333e20-11ed-4eeb-a311-1ad0cbb1a785 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:04.205099 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-bd333e20-11ed-4eeb-a311-1ad0cbb1a785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:04.277521 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17063dc4-a8fd-4a1b-8d66-590519ebdfb7 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:04.294864 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 361/709] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:04.314278 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:04.320056 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb3fa2-fd58-4503-b730-8522436207b0 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:04.320362 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb3fa2-fd58-4503-b730-8522436207b0 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:04.320715 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb3fa2-fd58-4503-b730-8522436207b0 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:04.324091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 362/710] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:04.325077 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098e1e23-5208-42cc-8e9e-534827197e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:04.326052 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e1e23-5208-42cc-8e9e-534827197e44 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:04.326538 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e1e23-5208-42cc-8e9e-534827197e44 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:04.327009 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e1e23-5208-42cc-8e9e-534827197e44 None admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:04.327114 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfb3fa2-fd58-4503-b730-8522436207b0 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:04.328091 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098e1e23-5208-42cc-8e9e-534827197e44 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:04.328865 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098e1e23-5208-42cc-8e9e-534827197e44 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:04.330595 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098e1e23-5208-42cc-8e9e-534827197e44 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:04.344254 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 349/711] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Jun 24 08:28:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:04.345916 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c233254a-f66e-4f04-8180-b6e39c876afa None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:04.346404 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c233254a-f66e-4f04-8180-b6e39c876afa None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:04.346897 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c233254a-f66e-4f04-8180-b6e39c876afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:04.370894 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-c233254a-f66e-4f04-8180-b6e39c876afa None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:04.372133 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-c233254a-f66e-4f04-8180-b6e39c876afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:04.424308 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:04.445240 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 350/712] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:04.446625 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c233254a-f66e-4f04-8180-b6e39c876afa None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:04.462531 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfb3fa2-fd58-4503-b730-8522436207b0 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:04.506258 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8bd92e-c23f-467d-9379-f3480c27d5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:04.507607 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8bd92e-c23f-467d-9379-f3480c27d5af None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:04.507889 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8bd92e-c23f-467d-9379-f3480c27d5af None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:04.508165 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8bd92e-c23f-467d-9379-f3480c27d5af None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:04.513716 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8bd92e-c23f-467d-9379-f3480c27d5af None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:04.513716 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8bd92e-c23f-467d-9379-f3480c27d5af None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:04.527801 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-3b8bd92e-c23f-467d-9379-f3480c27d5af None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 08:28:04.527801 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:04.527801 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 08:28:04.527801 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 08:28:04.527801 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 08:28:04.527801 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 08:28:04.527801 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 08:28:04.527801 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:28:04.527801 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 08:28:04.527801 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:04.527801 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:28:04.527801 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:04.527801 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:04.527801 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:04.527801 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:04.527801 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:04.529632 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:04.529632 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:04.529632 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:04.529632 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:04.529632 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:04.529632 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:04.529632 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:04.529632 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 08:28:04.529632 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 08:28:04.529632 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 08:28:04.529632 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 08:28:04.529632 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:04.529632 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:04.529632 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:04.529632 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:04.529632 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:04.533310 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:04.533310 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:04.533310 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:04.533310 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:04.533310 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:04.533310 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:04.533310 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:04.533310 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:04.533310 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:04.533310 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:04.533310 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:04.533310 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 08:28:04.533310 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:28:04.533310 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 08:28:04.533310 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:04.533310 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 363/713] 10.222.0.13 () {56 vars in 1129 bytes} [Fri Jun 24 08:28:04 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:28:04.536509 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8bd92e-c23f-467d-9379-f3480c27d5af None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:04.620077 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c233254a-f66e-4f04-8180-b6e39c876afa None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:04.641776 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8bd92e-c23f-467d-9379-f3480c27d5af None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:04.645730 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:04.681921 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e06978f-525a-4b5c-83b0-e5111d1b2289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:04.682383 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c53d1e7-ba6e-4ba8-a252-9cdb4743e6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:04.683032 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c53d1e7-ba6e-4ba8-a252-9cdb4743e6cf None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:04.683280 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e06978f-525a-4b5c-83b0-e5111d1b2289 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:04.683514 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c53d1e7-ba6e-4ba8-a252-9cdb4743e6cf None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:04.683766 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e06978f-525a-4b5c-83b0-e5111d1b2289 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:04.684000 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c53d1e7-ba6e-4ba8-a252-9cdb4743e6cf None admin] PATH_INFO: `/v3/services/network` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:04.684233 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e06978f-525a-4b5c-83b0-e5111d1b2289 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:04.684462 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c53d1e7-ba6e-4ba8-a252-9cdb4743e6cf None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:04.685558 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c53d1e7-ba6e-4ba8-a252-9cdb4743e6cf None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:04.685850 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e06978f-525a-4b5c-83b0-e5111d1b2289 None admin] RBAC: Adding query filter params (role.id=6ca1ae4292d9414ca6526c6e8608de9e, scope.project.id=180b0af5863349e99ae412b07c1e9465, user.id=bec8c2f113cf41a0939b81f38fbee813) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:04.686832 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e06978f-525a-4b5c-83b0-e5111d1b2289 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:04.688978 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e06978f-525a-4b5c-83b0-e5111d1b2289 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:04.698348 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-6c53d1e7-ba6e-4ba8-a252-9cdb4743e6cf None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 08:28:04.698348 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:04.698348 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 08:28:04.698348 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 08:28:04.698348 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 08:28:04.698348 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 08:28:04.698348 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 08:28:04.698348 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:28:04.698348 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 08:28:04.698348 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:04.698348 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:28:04.698348 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:04.698348 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:04.698348 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:04.698348 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:04.698348 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:04.698348 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:04.702035 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:04.702035 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:04.702035 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:04.702035 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:04.702035 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:04.702035 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:04.702035 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 08:28:04.702035 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 08:28:04.702035 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 08:28:04.702035 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 08:28:04.702035 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:04.702035 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:04.702035 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:04.702035 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:04.702035 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:04.702035 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:04.705032 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:04.705032 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:04.705032 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:04.705032 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:04.705032 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:04.705032 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:04.705032 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:04.705032 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:04.705032 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:04.705032 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:04.705032 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 08:28:04.705032 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:28:04.705032 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 08:28:04.705032 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:04.705032 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 364/714] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:28:04 2022] GET /identity/v3/services/network => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:04.711046 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c53d1e7-ba6e-4ba8-a252-9cdb4743e6cf None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:04.711046 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:04.712304 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 351/715] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Jun 24 08:28:04 2022] GET /identity/v3/role_assignments?user.id=bec8c2f113cf41a0939b81f38fbee813&scope.project.id=180b0af5863349e99ae412b07c1e9465&role.id=6ca1ae4292d9414ca6526c6e8608de9e => generated 608 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:04.712304 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e06978f-525a-4b5c-83b0-e5111d1b2289 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:04.816120 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c53d1e7-ba6e-4ba8-a252-9cdb4743e6cf None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:04.855688 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9445c17-0a95-4cb1-a3d0-ae025ab9beab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:04.856371 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9445c17-0a95-4cb1-a3d0-ae025ab9beab None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:04.856658 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9445c17-0a95-4cb1-a3d0-ae025ab9beab None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:04.856930 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9445c17-0a95-4cb1-a3d0-ae025ab9beab None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:04.857647 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9445c17-0a95-4cb1-a3d0-ae025ab9beab None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:04.857979 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9445c17-0a95-4cb1-a3d0-ae025ab9beab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:04.859369 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9445c17-0a95-4cb1-a3d0-ae025ab9beab None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:04.864885 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 365/716] 10.222.0.13 () {56 vars in 1124 bytes} [Fri Jun 24 08:28:04 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:04.878928 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9445c17-0a95-4cb1-a3d0-ae025ab9beab None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:04.945068 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e06978f-525a-4b5c-83b0-e5111d1b2289 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:04.983677 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa6043b-fa69-4d48-bc82-bc9b88e5c0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:04.984353 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa6043b-fa69-4d48-bc82-bc9b88e5c0a7 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:04.985086 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa6043b-fa69-4d48-bc82-bc9b88e5c0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:04.985381 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa6043b-fa69-4d48-bc82-bc9b88e5c0a7 None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:04.986125 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa6043b-fa69-4d48-bc82-bc9b88e5c0a7 None admin] RBAC: Adding query filter params (name=network) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:04.986435 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa6043b-fa69-4d48-bc82-bc9b88e5c0a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:04.988171 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa6043b-fa69-4d48-bc82-bc9b88e5c0a7 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:04.994408 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 352/717] 10.222.0.13 () {56 vars in 1112 bytes} [Fri Jun 24 08:28:04 2022] GET /identity/v3/services?name=network => generated 124 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:05.000737 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa6043b-fa69-4d48-bc82-bc9b88e5c0a7 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:05.005771 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:05.050273 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9445c17-0a95-4cb1-a3d0-ae025ab9beab None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:05.051214 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:05.090860 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14018a1-a77b-4782-b367-53862812de92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:05.091725 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14018a1-a77b-4782-b367-53862812de92 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:05.091837 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14018a1-a77b-4782-b367-53862812de92 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:05.092073 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14018a1-a77b-4782-b367-53862812de92 None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:05.092787 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14018a1-a77b-4782-b367-53862812de92 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:05.093096 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14018a1-a77b-4782-b367-53862812de92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:05.094511 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14018a1-a77b-4782-b367-53862812de92 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:05.101301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 366/718] 10.222.0.13 () {56 vars in 1124 bytes} [Fri Jun 24 08:28:04 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:05.119830 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa6043b-fa69-4d48-bc82-bc9b88e5c0a7 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:05.160280 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9675f2b9-64ac-4ec2-ae81-d1f4c05d70a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:05.161269 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9675f2b9-64ac-4ec2-ae81-d1f4c05d70a8 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:05.161779 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9675f2b9-64ac-4ec2-ae81-d1f4c05d70a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:05.162228 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9675f2b9-64ac-4ec2-ae81-d1f4c05d70a8 None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:05.163145 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9675f2b9-64ac-4ec2-ae81-d1f4c05d70a8 None admin] RBAC: Adding query filter params (type=network) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:05.163643 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9675f2b9-64ac-4ec2-ae81-d1f4c05d70a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:05.165734 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9675f2b9-64ac-4ec2-ae81-d1f4c05d70a8 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:05.172824 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 353/719] 10.222.0.13 () {56 vars in 1112 bytes} [Fri Jun 24 08:28:04 2022] GET /identity/v3/services?type=network => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:06.664217 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50269e8-f808-445f-969a-6c237da1107b None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:06.665511 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50269e8-f808-445f-969a-6c237da1107b None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:06.665511 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50269e8-f808-445f-969a-6c237da1107b None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:06.666498 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 367/720] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:06.671676 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec64fa-83f2-4c93-82e4-922d5a68f775 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:06.672171 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec64fa-83f2-4c93-82e4-922d5a68f775 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:06.672326 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec64fa-83f2-4c93-82e4-922d5a68f775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:06.694622 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-8cec64fa-83f2-4c93-82e4-922d5a68f775 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:06.696303 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-8cec64fa-83f2-4c93-82e4-922d5a68f775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:06.783694 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 354/721] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:06.803430 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f785bb-ccce-40ce-9f6a-8b1c0797da7b None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:06.803719 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f785bb-ccce-40ce-9f6a-8b1c0797da7b None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:06.804006 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f785bb-ccce-40ce-9f6a-8b1c0797da7b None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:06.806431 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 368/722] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:06.811335 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f70489-cfaf-4b6c-88a7-fd5ae17bee2b None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:06.811629 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f70489-cfaf-4b6c-88a7-fd5ae17bee2b None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:06.811906 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f70489-cfaf-4b6c-88a7-fd5ae17bee2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:06.813879 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f785bb-ccce-40ce-9f6a-8b1c0797da7b None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:06.830521 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-90f70489-cfaf-4b6c-88a7-fd5ae17bee2b None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:06.831689 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-90f70489-cfaf-4b6c-88a7-fd5ae17bee2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:06.889551 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:06.949903 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 355/723] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:06.973463 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f785bb-ccce-40ce-9f6a-8b1c0797da7b None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:06.994873 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f70489-cfaf-4b6c-88a7-fd5ae17bee2b None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:07.009528 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a891bfa9-2c37-44d2-8054-69507972fd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:07.010357 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a891bfa9-2c37-44d2-8054-69507972fd7f None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:07.018173 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:07.018680 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a891bfa9-2c37-44d2-8054-69507972fd7f None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:07.018987 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a891bfa9-2c37-44d2-8054-69507972fd7f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:07.042043 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a891bfa9-2c37-44d2-8054-69507972fd7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:07.049716 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-a891bfa9-2c37-44d2-8054-69507972fd7f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:28:07.049716 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:07.049716 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:07.049716 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:07.049716 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:07.049716 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:07.049716 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:07.049716 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:07.049716 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:07.049716 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:07.049716 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:07.049716 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:07.049716 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:07.049716 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:07.049716 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:07.049716 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:07.051772 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:07.051772 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:07.051772 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:07.051772 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:07.051772 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:07.051772 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:07.051772 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:07.051772 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:07.051772 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:07.051772 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:07.051772 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:07.051772 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:07.051772 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:07.051772 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:07.051772 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:07.051772 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:07.053781 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:07.053781 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:07.053781 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:07.053781 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:07.053781 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:07.053781 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:07.053781 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:07.053781 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:07.053781 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:07.053781 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:07.053781 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:28:07.053781 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:07.053781 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 369/724] 10.222.0.13 () {56 vars in 1102 bytes} [Fri Jun 24 08:28:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:28:07.063216 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a891bfa9-2c37-44d2-8054-69507972fd7f None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:07.163292 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f70489-cfaf-4b6c-88a7-fd5ae17bee2b None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:07.203266 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ee9949-1be4-45d4-8f95-3db652e62a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:07.205107 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee9949-1be4-45d4-8f95-3db652e62a3c None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:07.206064 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee9949-1be4-45d4-8f95-3db652e62a3c None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:07.206342 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee9949-1be4-45d4-8f95-3db652e62a3c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:07.226290 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ee9949-1be4-45d4-8f95-3db652e62a3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:07.243459 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-22ee9949-1be4-45d4-8f95-3db652e62a3c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:28:07.243459 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:07.243459 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:07.243459 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:07.243459 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:07.243459 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:07.243459 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:07.243459 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:07.243459 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:07.243459 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:07.243459 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:07.243459 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:07.243459 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:07.243459 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:07.243459 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:07.243459 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:07.245972 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:07.245972 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:07.245972 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:07.245972 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:07.245972 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:07.245972 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:07.245972 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:07.245972 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:07.245972 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:07.245972 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:07.245972 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:07.245972 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:07.245972 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:07.245972 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:07.245972 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:07.245972 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:07.247975 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:07.247975 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:07.247975 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:07.247975 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:07.247975 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:07.247975 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:07.247975 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:07.247975 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:07.247975 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:07.247975 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:07.247975 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:28:07.247975 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:07.247975 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 356/725] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Jun 24 08:28:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:07.253724 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ee9949-1be4-45d4-8f95-3db652e62a3c None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:07.269988 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a891bfa9-2c37-44d2-8054-69507972fd7f None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:07.310742 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f5e587-3b00-4b7f-9b31-e1c2abb3b03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:07.311407 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f5e587-3b00-4b7f-9b31-e1c2abb3b03e None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:07.311688 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f5e587-3b00-4b7f-9b31-e1c2abb3b03e None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:07.311964 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f5e587-3b00-4b7f-9b31-e1c2abb3b03e None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:07.312711 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f5e587-3b00-4b7f-9b31-e1c2abb3b03e None admin] RBAC: Adding query filter params (name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:07.313016 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f5e587-3b00-4b7f-9b31-e1c2abb3b03e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:07.314735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f5e587-3b00-4b7f-9b31-e1c2abb3b03e None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:07.323464 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:07.326366 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 370/726] 10.222.0.13 () {56 vars in 1103 bytes} [Fri Jun 24 08:28:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:07.338761 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f5e587-3b00-4b7f-9b31-e1c2abb3b03e None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:07.410295 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ee9949-1be4-45d4-8f95-3db652e62a3c None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:07.447093 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f72951-9045-46f3-9199-538dc0e90122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:07.447760 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f72951-9045-46f3-9199-538dc0e90122 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:07.448045 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f72951-9045-46f3-9199-538dc0e90122 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:07.448321 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f72951-9045-46f3-9199-538dc0e90122 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:07.449066 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f72951-9045-46f3-9199-538dc0e90122 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:07.449376 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f72951-9045-46f3-9199-538dc0e90122 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:07.450807 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f72951-9045-46f3-9199-538dc0e90122 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:07.465566 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:07.471149 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 357/727] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:28:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:07.479045 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f72951-9045-46f3-9199-538dc0e90122 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:07.561400 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f5e587-3b00-4b7f-9b31-e1c2abb3b03e None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:07.634259 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a19546e-3c05-4c8a-bda6-1a2981ce75e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:07.635094 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a19546e-3c05-4c8a-bda6-1a2981ce75e4 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:07.635427 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a19546e-3c05-4c8a-bda6-1a2981ce75e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:07.635904 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a19546e-3c05-4c8a-bda6-1a2981ce75e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:07.656497 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a19546e-3c05-4c8a-bda6-1a2981ce75e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:07.657993 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a19546e-3c05-4c8a-bda6-1a2981ce75e4 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:07.658654 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:07.671742 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f72951-9045-46f3-9199-538dc0e90122 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:07.677482 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-0a19546e-3c05-4c8a-bda6-1a2981ce75e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:07.677482 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:07.677482 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:07.677482 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:07.677482 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:07.677482 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:07.677482 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:07.677482 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:07.677482 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:07.677482 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:07.677482 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:07.677482 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:07.677482 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:28:07.677482 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:28:07.677482 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:28:07.678634 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:28:07.678634 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:07.678634 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:07.678634 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:07.678634 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:07.678634 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:07.678634 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:07.678634 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:07.678634 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:07.678634 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:07.678634 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:07.678634 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:07.678634 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:07.678634 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:07.678634 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:07.678634 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:07.678634 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:07.679560 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:28:07.679560 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:28:07.679560 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:07.679560 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:07.679560 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 371/728] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:07.681258 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f659b13-b2d6-4067-bbe0-a6531b368400 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:07.681258 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f659b13-b2d6-4067-bbe0-a6531b368400 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:07.681610 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f659b13-b2d6-4067-bbe0-a6531b368400 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:07.683302 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 372/729] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:07.684415 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaed63f-7ddc-4ca8-b318-ae74aae5d119 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:07.684702 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaed63f-7ddc-4ca8-b318-ae74aae5d119 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:07.684975 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaed63f-7ddc-4ca8-b318-ae74aae5d119 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:07.691663 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 373/730] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:07.691663 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaed63f-7ddc-4ca8-b318-ae74aae5d119 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:07.722158 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371d5bc8-33ab-4d68-9448-9fe8c4b64032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:07.722897 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d5bc8-33ab-4d68-9448-9fe8c4b64032 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:07.723177 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d5bc8-33ab-4d68-9448-9fe8c4b64032 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:07.723456 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d5bc8-33ab-4d68-9448-9fe8c4b64032 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:07.736938 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371d5bc8-33ab-4d68-9448-9fe8c4b64032 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:07.739002 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371d5bc8-33ab-4d68-9448-9fe8c4b64032 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:07.760357 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-371d5bc8-33ab-4d68-9448-9fe8c4b64032 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:07.760357 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:07.760357 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:07.760357 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:07.760357 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:07.760357 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:07.760357 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:07.760357 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:07.760357 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:07.760357 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:07.760357 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:07.760357 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:07.760357 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:28:07.760357 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:28:07.760357 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:28:07.761529 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:28:07.761529 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:07.761529 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:07.761529 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:07.761529 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:07.761529 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:07.761529 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:07.761529 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:07.761529 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:07.761529 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:07.761529 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:07.761529 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:07.761529 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:07.761529 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:07.761529 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:07.761529 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:07.761529 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:07.762660 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:28:07.762660 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:28:07.762660 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:07.762660 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:07.766172 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 358/731] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:07.767500 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a15e5a7-0393-4d19-8907-0313e05b5dac None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:07.767795 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a15e5a7-0393-4d19-8907-0313e05b5dac None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:07.768079 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a15e5a7-0393-4d19-8907-0313e05b5dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:07.784601 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-3a15e5a7-0393-4d19-8907-0313e05b5dac None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:07.785822 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-3a15e5a7-0393-4d19-8907-0313e05b5dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:07.808253 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaed63f-7ddc-4ca8-b318-ae74aae5d119 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:07.810556 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:07.843447 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26345df-4c54-4d15-9ddd-8f7a19aa8ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:07.844104 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26345df-4c54-4d15-9ddd-8f7a19aa8ba0 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:07.844396 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26345df-4c54-4d15-9ddd-8f7a19aa8ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:07.844669 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26345df-4c54-4d15-9ddd-8f7a19aa8ba0 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:07.845384 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26345df-4c54-4d15-9ddd-8f7a19aa8ba0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:07.845709 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26345df-4c54-4d15-9ddd-8f7a19aa8ba0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:07.847079 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26345df-4c54-4d15-9ddd-8f7a19aa8ba0 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:07.852880 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 359/732] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:07.853998 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef3712-9341-4dd8-ab57-0ffb85f6c357 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:07.854361 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef3712-9341-4dd8-ab57-0ffb85f6c357 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:07.854647 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef3712-9341-4dd8-ab57-0ffb85f6c357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:07.868243 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 374/733] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:28:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:07.869210 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26345df-4c54-4d15-9ddd-8f7a19aa8ba0 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:07.875218 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-9bef3712-9341-4dd8-ab57-0ffb85f6c357 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:07.876636 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-9bef3712-9341-4dd8-ab57-0ffb85f6c357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:07.938899 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:07.943547 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 360/734] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:07.945089 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaeccd3-6323-417f-a863-5ea19a4322a8 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:07.945685 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaeccd3-6323-417f-a863-5ea19a4322a8 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:07.946447 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaeccd3-6323-417f-a863-5ea19a4322a8 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:07.947973 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 361/735] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:07.948520 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaeccd3-6323-417f-a863-5ea19a4322a8 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:07.980170 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26345df-4c54-4d15-9ddd-8f7a19aa8ba0 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:08.014936 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4bf0ff-1b8d-48d6-83cb-0c1455b78301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:08.015609 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4bf0ff-1b8d-48d6-83cb-0c1455b78301 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:08.015900 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4bf0ff-1b8d-48d6-83cb-0c1455b78301 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:08.016178 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4bf0ff-1b8d-48d6-83cb-0c1455b78301 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:08.016899 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4bf0ff-1b8d-48d6-83cb-0c1455b78301 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:08.017206 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4bf0ff-1b8d-48d6-83cb-0c1455b78301 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:08.018605 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4bf0ff-1b8d-48d6-83cb-0c1455b78301 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:08.030659 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 375/736] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:28:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:08.031478 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4bf0ff-1b8d-48d6-83cb-0c1455b78301 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:08.059890 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaeccd3-6323-417f-a863-5ea19a4322a8 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:08.062929 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:08.100449 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbafd70-c9e9-4fcd-8f4a-c4d882c0d413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:08.101262 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbafd70-c9e9-4fcd-8f4a-c4d882c0d413 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:08.101549 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbafd70-c9e9-4fcd-8f4a-c4d882c0d413 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:08.101858 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbafd70-c9e9-4fcd-8f4a-c4d882c0d413 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:08.109955 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbafd70-c9e9-4fcd-8f4a-c4d882c0d413 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:08.111414 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbafd70-c9e9-4fcd-8f4a-c4d882c0d413 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:08.127747 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-fdbafd70-c9e9-4fcd-8f4a-c4d882c0d413 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 08:28:08.127747 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:08.127747 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:08.127747 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:08.127747 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:08.127747 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:08.127747 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:08.127747 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:08.127747 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:08.127747 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:08.127747 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:08.127747 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:08.127747 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:28:08.127747 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:28:08.127747 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:28:08.127747 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:28:08.128861 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:08.128861 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:08.128861 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:28:08.128861 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:28:08.128861 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:28:08.128861 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:28:08.128861 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:08.128861 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:08.128861 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:08.128861 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:08.128861 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:08.128861 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:08.128861 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:08.128861 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:08.128861 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:08.128861 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:08.128861 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:08.128861 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:08.134874 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:08.134874 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:08.134874 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:28:08.134874 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:28:08.134874 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:28:08.134874 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:28:08.134874 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:28:08.134874 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:28:08.134874 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 08:28:08.134874 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:08.134874 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 362/737] 10.222.0.13 () {56 vars in 1117 bytes} [Fri Jun 24 08:28:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:08.134874 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca5dd9-586c-4b89-8ff2-02b6b1a62eb8 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:08.134874 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca5dd9-586c-4b89-8ff2-02b6b1a62eb8 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:08.135591 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca5dd9-586c-4b89-8ff2-02b6b1a62eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:08.142397 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4bf0ff-1b8d-48d6-83cb-0c1455b78301 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:08.147730 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-0cca5dd9-586c-4b89-8ff2-02b6b1a62eb8 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:08.148978 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-0cca5dd9-586c-4b89-8ff2-02b6b1a62eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:08.177249 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4b85ef-99a4-4a21-8d09-a9b821c386e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:08.178173 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b85ef-99a4-4a21-8d09-a9b821c386e5 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:08.178454 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b85ef-99a4-4a21-8d09-a9b821c386e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:08.178734 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b85ef-99a4-4a21-8d09-a9b821c386e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:08.189192 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4b85ef-99a4-4a21-8d09-a9b821c386e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:08.196684 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-0e4b85ef-99a4-4a21-8d09-a9b821c386e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:28:08.196684 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:08.196684 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:08.196684 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:08.196684 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:08.196684 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:08.196684 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:08.196684 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:08.196684 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:08.196684 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:08.196684 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:08.196684 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:08.196684 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:08.196684 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:08.196684 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:08.196684 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:08.197835 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:08.197835 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:08.197835 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:08.197835 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:08.197835 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:08.197835 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:08.197835 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:08.197835 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:08.197835 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:08.197835 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:08.197835 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:08.197835 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:08.197835 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:08.197835 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:08.197835 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:08.197835 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:08.198683 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:08.198683 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:08.198683 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:08.198683 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:08.198683 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:08.198683 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:08.198683 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:08.198683 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:08.198683 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:08.198683 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:08.198683 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:28:08.198683 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:08.198683 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 376/738] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Jun 24 08:28:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:08.199377 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4b85ef-99a4-4a21-8d09-a9b821c386e5 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:08.206318 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:08.218623 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 363/739] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:08.219428 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cca5dd9-586c-4b89-8ff2-02b6b1a62eb8 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:08.299673 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:08.315643 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4b85ef-99a4-4a21-8d09-a9b821c386e5 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:08.320139 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cca5dd9-586c-4b89-8ff2-02b6b1a62eb8 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:08.349094 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53456d6-8595-4522-8d8a-90bb2977cb87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:08.349775 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53456d6-8595-4522-8d8a-90bb2977cb87 None admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:08.350058 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53456d6-8595-4522-8d8a-90bb2977cb87 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:08.350330 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53456d6-8595-4522-8d8a-90bb2977cb87 None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:08.351040 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53456d6-8595-4522-8d8a-90bb2977cb87 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:08.353008 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53456d6-8595-4522-8d8a-90bb2977cb87 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:08.357333 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adaef38-e98e-4673-873d-1162da4ecdc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:08.358163 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adaef38-e98e-4673-873d-1162da4ecdc1 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:08.358443 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adaef38-e98e-4673-873d-1162da4ecdc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:08.358720 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adaef38-e98e-4673-873d-1162da4ecdc1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:08.366406 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adaef38-e98e-4673-873d-1162da4ecdc1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:08.373101 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 377/740] 10.222.0.13 () {60 vars in 1141 bytes} [Fri Jun 24 08:28:08 2022] POST /identity/v3/services => generated 251 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:28:08.374230 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adaef38-e98e-4673-873d-1162da4ecdc1 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:08.374326 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53456d6-8595-4522-8d8a-90bb2977cb87 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:08.390398 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-0adaef38-e98e-4673-873d-1162da4ecdc1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 08:28:08.390398 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:08.390398 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:08.390398 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:08.390398 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:08.390398 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:08.390398 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:08.390398 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:08.390398 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:08.390398 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:08.390398 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:08.390398 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:08.390398 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:28:08.390398 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:28:08.390398 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:28:08.390398 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:28:08.391490 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:08.391490 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:08.391490 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:28:08.391490 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:28:08.391490 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:28:08.391490 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:28:08.391490 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:08.391490 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:08.391490 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:08.391490 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:08.391490 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:08.391490 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:08.391490 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:08.391490 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:08.391490 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:08.391490 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:08.391490 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:08.391490 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:08.397563 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:08.397563 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:08.397563 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:28:08.397563 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:28:08.397563 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:28:08.397563 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:28:08.397563 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:28:08.397563 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:28:08.397563 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 08:28:08.397563 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:08.397563 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 364/741] 10.222.0.13 () {56 vars in 1093 bytes} [Fri Jun 24 08:28:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:08.397563 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adaef38-e98e-4673-873d-1162da4ecdc1 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:08.466178 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:08.526433 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adaef38-e98e-4673-873d-1162da4ecdc1 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:08.529672 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53456d6-8595-4522-8d8a-90bb2977cb87 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:08.566139 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab86a4d4-7936-4ee2-b495-a6bcb049a830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:08.566743 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86a4d4-7936-4ee2-b495-a6bcb049a830 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:08.567026 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86a4d4-7936-4ee2-b495-a6bcb049a830 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:08.567302 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86a4d4-7936-4ee2-b495-a6bcb049a830 None admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:08.568140 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab86a4d4-7936-4ee2-b495-a6bcb049a830 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:08.568450 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab86a4d4-7936-4ee2-b495-a6bcb049a830 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:08.569893 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab86a4d4-7936-4ee2-b495-a6bcb049a830 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:08.570211 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6bb1ee-3aa0-4ab8-bd5d-285148f9c020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:08.571451 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6bb1ee-3aa0-4ab8-bd5d-285148f9c020 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:08.571738 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6bb1ee-3aa0-4ab8-bd5d-285148f9c020 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:08.572021 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6bb1ee-3aa0-4ab8-bd5d-285148f9c020 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:08.572961 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6bb1ee-3aa0-4ab8-bd5d-285148f9c020 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:08.573998 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6bb1ee-3aa0-4ab8-bd5d-285148f9c020 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:08.575370 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6bb1ee-3aa0-4ab8-bd5d-285148f9c020 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:08.583432 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:08.590221 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 365/742] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:28:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:08.590221 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6bb1ee-3aa0-4ab8-bd5d-285148f9c020 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:08.595247 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 378/743] 10.222.0.13 () {56 vars in 1149 bytes} [Fri Jun 24 08:28:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:08.596086 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab86a4d4-7936-4ee2-b495-a6bcb049a830 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:08.700633 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6bb1ee-3aa0-4ab8-bd5d-285148f9c020 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:08.706368 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab86a4d4-7936-4ee2-b495-a6bcb049a830 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:08.716008 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:08.742430 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8f9fae-1182-4afe-834e-d346c200591c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:08.743274 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8f9fae-1182-4afe-834e-d346c200591c None admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:08.743564 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8f9fae-1182-4afe-834e-d346c200591c None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:08.743854 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8f9fae-1182-4afe-834e-d346c200591c None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:08.744563 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8f9fae-1182-4afe-834e-d346c200591c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:08.745983 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8f9fae-1182-4afe-834e-d346c200591c None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:08.748204 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01458da1-bf69-424d-9e56-5c6c045920d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:08.749070 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01458da1-bf69-424d-9e56-5c6c045920d2 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:08.749526 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01458da1-bf69-424d-9e56-5c6c045920d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:08.749998 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01458da1-bf69-424d-9e56-5c6c045920d2 None admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:08.750835 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01458da1-bf69-424d-9e56-5c6c045920d2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:08.751270 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01458da1-bf69-424d-9e56-5c6c045920d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:08.752998 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01458da1-bf69-424d-9e56-5c6c045920d2 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:08.762974 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 366/744] 10.222.0.13 () {60 vars in 1141 bytes} [Fri Jun 24 08:28:08 2022] POST /identity/v3/services => generated 240 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:28:08.763954 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8f9fae-1182-4afe-834e-d346c200591c None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:08.791733 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 379/745] 10.222.0.13 () {56 vars in 1133 bytes} [Fri Jun 24 08:28:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:08.792871 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01458da1-bf69-424d-9e56-5c6c045920d2 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:08.873593 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:08.896540 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8f9fae-1182-4afe-834e-d346c200591c None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:08.936567 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78fcc57-30f5-4f26-b9c3-62058c9ff7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:08.936832 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01458da1-bf69-424d-9e56-5c6c045920d2 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:08.937839 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78fcc57-30f5-4f26-b9c3-62058c9ff7b3 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:08.938274 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78fcc57-30f5-4f26-b9c3-62058c9ff7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:08.938692 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78fcc57-30f5-4f26-b9c3-62058c9ff7b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:08.953486 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78fcc57-30f5-4f26-b9c3-62058c9ff7b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:08.955352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78fcc57-30f5-4f26-b9c3-62058c9ff7b3 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:08.979428 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-b78fcc57-30f5-4f26-b9c3-62058c9ff7b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:08.979428 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:08.979428 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:08.979428 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:08.979428 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:08.979428 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:08.979428 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:08.979428 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:08.979428 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:08.979428 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:08.979428 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:08.979428 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:08.979428 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:28:08.979428 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:28:08.979428 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:28:08.980352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:28:08.980352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:08.980352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:08.980352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:08.980352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:08.980352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:08.980352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:08.980352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:08.980352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:08.980352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:08.980352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:08.980352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:08.980352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:08.980352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:08.980352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:08.980352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:08.980352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:08.981233 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:28:08.981233 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:28:08.981233 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:08.981233 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:08.981233 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 367/746] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:08.981502 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78fcc57-30f5-4f26-b9c3-62058c9ff7b3 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:08.986079 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0c56dc-1150-4464-8c5a-df0c7d833316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:08.987000 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c56dc-1150-4464-8c5a-df0c7d833316 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:08.987734 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c56dc-1150-4464-8c5a-df0c7d833316 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:08.988215 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c56dc-1150-4464-8c5a-df0c7d833316 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:09.005551 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:09.089437 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0c56dc-1150-4464-8c5a-df0c7d833316 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:09.091519 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0c56dc-1150-4464-8c5a-df0c7d833316 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:09.103265 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78fcc57-30f5-4f26-b9c3-62058c9ff7b3 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:09.140395 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac8107-7829-4524-94b7-af21cacc36c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:09.141507 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac8107-7829-4524-94b7-af21cacc36c7 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:09.142046 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac8107-7829-4524-94b7-af21cacc36c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:09.142519 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac8107-7829-4524-94b7-af21cacc36c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:09.155247 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afac8107-7829-4524-94b7-af21cacc36c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:09.156999 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afac8107-7829-4524-94b7-af21cacc36c7 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:09.167707 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:09.178091 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-afac8107-7829-4524-94b7-af21cacc36c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:09.178091 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:09.178091 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:09.178091 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:09.178091 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:09.178091 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:09.178091 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:09.178091 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:09.178091 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:09.178091 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:09.178091 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:09.178091 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:09.178091 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:28:09.178091 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:28:09.178091 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:28:09.178983 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:28:09.178983 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:09.178983 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:09.178983 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:09.178983 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:09.178983 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:09.178983 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:09.178983 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:09.178983 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:09.178983 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:09.178983 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:09.178983 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:09.178983 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:09.178983 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:09.178983 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:09.178983 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:09.178983 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:09.179865 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:28:09.179865 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:28:09.179865 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:09.179865 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:09.186658 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 368/747] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:09.187753 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac8107-7829-4524-94b7-af21cacc36c7 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:09.233203 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 380/748] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:28:08 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 08:28:09.234097 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0c56dc-1150-4464-8c5a-df0c7d833316 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:09.271678 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:09.306736 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac8107-7829-4524-94b7-af21cacc36c7 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:09.344858 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0273bf-5b59-4265-9d9e-f6645a53a5f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:09.346009 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0273bf-5b59-4265-9d9e-f6645a53a5f5 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:09.346537 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0273bf-5b59-4265-9d9e-f6645a53a5f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:09.347020 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0273bf-5b59-4265-9d9e-f6645a53a5f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:09.348006 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0273bf-5b59-4265-9d9e-f6645a53a5f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:09.348652 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0273bf-5b59-4265-9d9e-f6645a53a5f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:09.350433 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0273bf-5b59-4265-9d9e-f6645a53a5f5 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:09.368937 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0c56dc-1150-4464-8c5a-df0c7d833316 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:09.369669 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 369/749] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:28:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:09.370822 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0273bf-5b59-4265-9d9e-f6645a53a5f5 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:09.409382 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7237e47f-f7c8-4fae-a8b0-75583062f51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:09.410441 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237e47f-f7c8-4fae-a8b0-75583062f51f None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:09.410931 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237e47f-f7c8-4fae-a8b0-75583062f51f None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:09.411399 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237e47f-f7c8-4fae-a8b0-75583062f51f None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:09.412304 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7237e47f-f7c8-4fae-a8b0-75583062f51f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:09.412801 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7237e47f-f7c8-4fae-a8b0-75583062f51f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:09.415030 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7237e47f-f7c8-4fae-a8b0-75583062f51f None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:09.433447 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 381/750] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:28:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:09.434655 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7237e47f-f7c8-4fae-a8b0-75583062f51f None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:09.454530 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:09.494157 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0273bf-5b59-4265-9d9e-f6645a53a5f5 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:09.534775 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:09.540917 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832a1584-97bc-4dd8-8140-8eda75acb506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:09.542539 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a1584-97bc-4dd8-8140-8eda75acb506 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:09.543263 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a1584-97bc-4dd8-8140-8eda75acb506 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:09.543692 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a1584-97bc-4dd8-8140-8eda75acb506 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:09.558323 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832a1584-97bc-4dd8-8140-8eda75acb506 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:09.558896 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7237e47f-f7c8-4fae-a8b0-75583062f51f None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:09.560221 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832a1584-97bc-4dd8-8140-8eda75acb506 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:09.583228 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-832a1584-97bc-4dd8-8140-8eda75acb506 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:09.583228 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:09.583228 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:09.583228 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:09.583228 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:09.583228 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:09.583228 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:09.583228 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:09.583228 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:09.583228 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:09.583228 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:09.583228 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:09.583228 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:28:09.583228 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:28:09.583228 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:28:09.584351 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:28:09.584351 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:09.584351 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:09.584351 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:09.584351 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:09.584351 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:09.584351 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:09.584351 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:09.584351 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:09.584351 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:09.584351 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:09.584351 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:09.584351 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:09.584351 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:09.584351 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:09.584351 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:09.584351 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:09.585229 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:28:09.585229 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:28:09.585229 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:09.585229 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:09.585796 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 370/751] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:09.587201 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832a1584-97bc-4dd8-8140-8eda75acb506 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:09.606917 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa6c86-5815-4557-a2d5-e66b23f9dbc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:09.608377 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa6c86-5815-4557-a2d5-e66b23f9dbc9 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:09.608842 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa6c86-5815-4557-a2d5-e66b23f9dbc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:09.609314 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa6c86-5815-4557-a2d5-e66b23f9dbc9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:09.621068 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafa6c86-5815-4557-a2d5-e66b23f9dbc9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:09.623337 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafa6c86-5815-4557-a2d5-e66b23f9dbc9 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:09.645413 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-aafa6c86-5815-4557-a2d5-e66b23f9dbc9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 08:28:09.645413 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:09.645413 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:09.645413 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:09.645413 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:09.645413 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:09.645413 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:09.645413 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:09.645413 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:09.645413 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:09.645413 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:09.645413 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:09.645413 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:28:09.645413 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:28:09.645413 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:28:09.645413 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:28:09.646459 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:09.646459 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:09.646459 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:28:09.646459 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:28:09.646459 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:28:09.646459 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:28:09.646459 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:09.646459 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:09.646459 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:09.646459 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:09.646459 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:09.646459 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:09.646459 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:09.646459 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:09.646459 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:09.646459 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:09.646459 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:09.646459 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:09.647425 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:09.647425 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:09.647425 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:28:09.647425 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:28:09.647425 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:28:09.647425 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:28:09.647425 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:28:09.647425 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:28:09.647425 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 08:28:09.647425 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:09.655454 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 382/752] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Jun 24 08:28:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:09.656500 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa6c86-5815-4557-a2d5-e66b23f9dbc9 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:09.703674 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832a1584-97bc-4dd8-8140-8eda75acb506 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:09.741097 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:09.743763 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a3a175-b168-488c-9e52-edaeb60ffa32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:09.745005 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3a175-b168-488c-9e52-edaeb60ffa32 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:09.745314 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3a175-b168-488c-9e52-edaeb60ffa32 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:09.745613 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3a175-b168-488c-9e52-edaeb60ffa32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:09.754245 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a3a175-b168-488c-9e52-edaeb60ffa32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:09.755705 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a3a175-b168-488c-9e52-edaeb60ffa32 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:09.769884 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa6c86-5815-4557-a2d5-e66b23f9dbc9 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:09.771220 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-54a3a175-b168-488c-9e52-edaeb60ffa32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:28:09.771220 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:09.771220 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:09.771220 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:09.771220 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:09.771220 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:09.771220 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:09.771220 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:09.771220 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:09.771220 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:09.771220 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:09.771220 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:09.771220 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:28:09.771220 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:28:09.771220 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:28:09.772085 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:28:09.772085 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:09.772085 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:09.772085 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:09.772085 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:09.772085 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:09.772085 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:09.772085 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:09.772085 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:09.772085 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:09.772085 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:09.772085 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:09.772085 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:09.772085 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:09.772085 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:09.772085 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:09.772085 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:09.772971 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:28:09.772971 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:28:09.772971 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:28:09.772971 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:28:09.772971 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:28:09.772971 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:28:09.772971 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:28:09.772971 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:09.777755 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 371/753] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:28:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:09.778595 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a3a175-b168-488c-9e52-edaeb60ffa32 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:09.809942 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3394fd50-5f4c-45bc-b9a1-e1ab0ec53292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:09.810985 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3394fd50-5f4c-45bc-b9a1-e1ab0ec53292 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:09.811489 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3394fd50-5f4c-45bc-b9a1-e1ab0ec53292 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:09.811960 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3394fd50-5f4c-45bc-b9a1-e1ab0ec53292 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:09.812893 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3394fd50-5f4c-45bc-b9a1-e1ab0ec53292 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:09.813392 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3394fd50-5f4c-45bc-b9a1-e1ab0ec53292 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:09.814977 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3394fd50-5f4c-45bc-b9a1-e1ab0ec53292 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:09.828146 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 383/754] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:28:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:09.828862 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:09.829431 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3394fd50-5f4c-45bc-b9a1-e1ab0ec53292 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:09.890899 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a3a175-b168-488c-9e52-edaeb60ffa32 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:09.930904 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae498d-e79f-41d8-808d-a133939514f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:09.931867 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae498d-e79f-41d8-808d-a133939514f0 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:09.932069 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae498d-e79f-41d8-808d-a133939514f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:09.932233 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae498d-e79f-41d8-808d-a133939514f0 None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:09.933051 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dae498d-e79f-41d8-808d-a133939514f0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:09.933378 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dae498d-e79f-41d8-808d-a133939514f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:09.934866 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dae498d-e79f-41d8-808d-a133939514f0 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:09.935780 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3394fd50-5f4c-45bc-b9a1-e1ab0ec53292 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:09.959009 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 372/755] 10.222.0.13 () {56 vars in 1135 bytes} [Fri Jun 24 08:28:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:09.959849 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae498d-e79f-41d8-808d-a133939514f0 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:09.978976 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e3e319-a04f-41a0-b338-990e22a84e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:09.979934 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e3e319-a04f-41a0-b338-990e22a84e50 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:09.980209 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e3e319-a04f-41a0-b338-990e22a84e50 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:09.980492 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e3e319-a04f-41a0-b338-990e22a84e50 None admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:09.981425 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e3e319-a04f-41a0-b338-990e22a84e50 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:09.982037 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e3e319-a04f-41a0-b338-990e22a84e50 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:09.983565 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e3e319-a04f-41a0-b338-990e22a84e50 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:09.987375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:10.003874 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 384/756] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Jun 24 08:28:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:10.004944 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e3e319-a04f-41a0-b338-990e22a84e50 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:10.091938 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae498d-e79f-41d8-808d-a133939514f0 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:10.093950 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:10.141387 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150b495a-6a03-4683-b43a-c9ffddda0cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:10.142314 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b495a-6a03-4683-b43a-c9ffddda0cee None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:10.142642 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b495a-6a03-4683-b43a-c9ffddda0cee None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:10.142994 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b495a-6a03-4683-b43a-c9ffddda0cee None admin] PATH_INFO: `/v3/projects/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:10.145161 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e3e319-a04f-41a0-b338-990e22a84e50 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:10.153561 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150b495a-6a03-4683-b43a-c9ffddda0cee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:10.154541 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150b495a-6a03-4683-b43a-c9ffddda0cee None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:10.168725 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-150b495a-6a03-4683-b43a-c9ffddda0cee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:28:10.168725 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:10.168725 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:10.168725 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:10.168725 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:10.168725 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:10.168725 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:10.168725 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:10.168725 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:10.168725 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:10.168725 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:10.168725 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:10.168725 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:28:10.168725 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:28:10.168725 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:28:10.169625 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:28:10.169625 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:10.169625 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:10.169625 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:10.169625 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:10.169625 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:10.169625 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:10.169625 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:10.169625 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:10.169625 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:10.169625 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:10.169625 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:10.169625 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:10.169625 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:10.169625 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:10.169625 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:10.169625 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:10.170546 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:28:10.170546 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:28:10.170546 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:28:10.170546 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:28:10.170546 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:28:10.170546 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:28:10.170546 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:28:10.170546 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:10.170546 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 373/757] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:28:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:10.171655 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150b495a-6a03-4683-b43a-c9ffddda0cee None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:10.198613 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb454dd-fda2-4a31-a6fc-b34d46833032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:10.199773 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb454dd-fda2-4a31-a6fc-b34d46833032 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:10.200523 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb454dd-fda2-4a31-a6fc-b34d46833032 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:10.201051 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb454dd-fda2-4a31-a6fc-b34d46833032 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:10.280568 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150b495a-6a03-4683-b43a-c9ffddda0cee None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:10.290660 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:10.301439 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb454dd-fda2-4a31-a6fc-b34d46833032 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:10.303702 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb454dd-fda2-4a31-a6fc-b34d46833032 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:10.318067 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021ff70e-979d-4ad2-a000-9084c263bd0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:10.319044 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ff70e-979d-4ad2-a000-9084c263bd0b None admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:10.319403 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ff70e-979d-4ad2-a000-9084c263bd0b None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:10.319753 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ff70e-979d-4ad2-a000-9084c263bd0b None admin] PATH_INFO: `/v3/projects/180b0af5863349e99ae412b07c1e9465/users/74bb1bc6821b4500b26f160ccae5b7d5/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:10.322424 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021ff70e-979d-4ad2-a000-9084c263bd0b None admin] RBAC: Authorizing `identity:create_grant(project_id=180b0af5863349e99ae412b07c1e9465, user_id=74bb1bc6821b4500b26f160ccae5b7d5, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:10.324000 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021ff70e-979d-4ad2-a000-9084c263bd0b None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:10.345793 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 374/758] 10.222.0.13 () {58 vars in 1439 bytes} [Fri Jun 24 08:28:10 2022] PUT /identity/v3/projects/180b0af5863349e99ae412b07c1e9465/users/74bb1bc6821b4500b26f160ccae5b7d5/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:28:10.347982 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021ff70e-979d-4ad2-a000-9084c263bd0b None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:10.410336 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:10.490960 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 385/759] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:28:10 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 08:28:10.492396 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b95f2b-e205-4308-bd05-05f0f4ec3d0f None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:10.493172 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b95f2b-e205-4308-bd05-05f0f4ec3d0f None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:10.493720 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b95f2b-e205-4308-bd05-05f0f4ec3d0f None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:10.495838 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 386/760] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:10.499084 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b95f2b-e205-4308-bd05-05f0f4ec3d0f None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:10.537929 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021ff70e-979d-4ad2-a000-9084c263bd0b None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:10.587158 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860ac836-8560-4e9f-99f7-c180d6c92f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:10.587931 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ac836-8560-4e9f-99f7-c180d6c92f35 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:10.588213 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ac836-8560-4e9f-99f7-c180d6c92f35 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:10.588493 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ac836-8560-4e9f-99f7-c180d6c92f35 None admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:10.589281 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860ac836-8560-4e9f-99f7-c180d6c92f35 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:10.589591 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860ac836-8560-4e9f-99f7-c180d6c92f35 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:10.591054 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860ac836-8560-4e9f-99f7-c180d6c92f35 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:10.607400 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 375/761] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Jun 24 08:28:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:10.609788 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f2544-9901-43dd-9298-ec6b3e34db1d None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:10.610293 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f2544-9901-43dd-9298-ec6b3e34db1d None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:10.611026 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f2544-9901-43dd-9298-ec6b3e34db1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:10.628618 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:10.638177 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-5c8f2544-9901-43dd-9298-ec6b3e34db1d None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:10.638961 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b95f2b-e205-4308-bd05-05f0f4ec3d0f None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:10.639444 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-5c8f2544-9901-43dd-9298-ec6b3e34db1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:10.676727 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726b6e5c-1a52-4236-8625-e50e836719f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:10.677995 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726b6e5c-1a52-4236-8625-e50e836719f2 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:10.678510 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726b6e5c-1a52-4236-8625-e50e836719f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:10.678994 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726b6e5c-1a52-4236-8625-e50e836719f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:10.691645 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726b6e5c-1a52-4236-8625-e50e836719f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:10.693407 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726b6e5c-1a52-4236-8625-e50e836719f2 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:10.712876 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-726b6e5c-1a52-4236-8625-e50e836719f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:10.712876 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:10.712876 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:10.712876 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:10.712876 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:10.712876 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:10.712876 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:10.712876 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:10.712876 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:10.712876 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:10.712876 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:10.712876 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:10.712876 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:28:10.712876 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:28:10.712876 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:28:10.713857 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:28:10.713857 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:10.713857 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:10.713857 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:10.713857 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:10.713857 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:10.713857 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:10.713857 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:10.713857 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:10.713857 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:10.713857 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:10.713857 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:10.713857 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:10.713857 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:10.713857 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:10.713857 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:10.713857 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:10.714761 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:28:10.714761 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:28:10.714761 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:10.714761 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:10.717023 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 387/762] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:10.721737 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 376/763] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:28:10.722364 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726b6e5c-1a52-4236-8625-e50e836719f2 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:10.725887 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8f2544-9901-43dd-9298-ec6b3e34db1d None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:10.742145 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:10.834120 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726b6e5c-1a52-4236-8625-e50e836719f2 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:10.839877 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8f2544-9901-43dd-9298-ec6b3e34db1d None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:10.873902 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb966b5-0f9e-4e3c-ba23-f3c2c96c9870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:10.874948 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb966b5-0f9e-4e3c-ba23-f3c2c96c9870 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:10.875469 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb966b5-0f9e-4e3c-ba23-f3c2c96c9870 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:10.875990 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb966b5-0f9e-4e3c-ba23-f3c2c96c9870 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:10.877323 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb966b5-0f9e-4e3c-ba23-f3c2c96c9870 None admin] RBAC: Adding query filter params (role.id=6ca1ae4292d9414ca6526c6e8608de9e, scope.project.id=180b0af5863349e99ae412b07c1e9465, user.id=85d9c4c57681447c898ebfd7990b05b6) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:10.877764 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb966b5-0f9e-4e3c-ba23-f3c2c96c9870 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:10.879330 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb966b5-0f9e-4e3c-ba23-f3c2c96c9870 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:10.881011 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb367fc-5cd8-418c-ae19-a779f99c748c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:10.881927 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb367fc-5cd8-418c-ae19-a779f99c748c None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:10.882354 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb367fc-5cd8-418c-ae19-a779f99c748c None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:10.882773 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb367fc-5cd8-418c-ae19-a779f99c748c None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:10.884313 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb367fc-5cd8-418c-ae19-a779f99c748c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:10.884843 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb367fc-5cd8-418c-ae19-a779f99c748c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:10.886450 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb367fc-5cd8-418c-ae19-a779f99c748c None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:10.886910 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 388/764] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Jun 24 08:28:10 2022] GET /identity/v3/role_assignments?user.id=85d9c4c57681447c898ebfd7990b05b6&scope.project.id=180b0af5863349e99ae412b07c1e9465&role.id=6ca1ae4292d9414ca6526c6e8608de9e => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:10.887866 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb966b5-0f9e-4e3c-ba23-f3c2c96c9870 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:10.901572 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 377/765] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:28:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:10.907447 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb367fc-5cd8-418c-ae19-a779f99c748c None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:10.910937 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:10.993890 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:11.027747 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb966b5-0f9e-4e3c-ba23-f3c2c96c9870 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:11.034343 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb367fc-5cd8-418c-ae19-a779f99c748c None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:11.070364 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765929cd-da6e-422b-b4eb-259449941fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:11.072565 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765929cd-da6e-422b-b4eb-259449941fbb None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:11.072856 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765929cd-da6e-422b-b4eb-259449941fbb None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:11.073552 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765929cd-da6e-422b-b4eb-259449941fbb None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:11.074505 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765929cd-da6e-422b-b4eb-259449941fbb None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:11.076115 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765929cd-da6e-422b-b4eb-259449941fbb None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:11.082436 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b212ad15-cbf6-4abf-817a-ee469b3089a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:11.083355 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212ad15-cbf6-4abf-817a-ee469b3089a1 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:11.083724 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212ad15-cbf6-4abf-817a-ee469b3089a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:11.084090 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212ad15-cbf6-4abf-817a-ee469b3089a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:11.092107 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-765929cd-da6e-422b-b4eb-259449941fbb None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 08:28:11.092107 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:11.092107 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 08:28:11.092107 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 08:28:11.092107 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 08:28:11.092107 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 08:28:11.092107 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 08:28:11.092107 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:28:11.092107 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 08:28:11.092107 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:11.092107 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:28:11.092107 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:11.092107 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:11.092107 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:11.092107 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:11.092107 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:11.093039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:11.093039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:11.093039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:11.093039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:11.093039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:11.093039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:11.093039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:11.093039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 08:28:11.093039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 08:28:11.093039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 08:28:11.093039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 08:28:11.093039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:11.093039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:11.093039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:11.093039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:11.093039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:11.093963 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:11.093963 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:11.093963 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:11.093963 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:11.093963 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:11.093963 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:11.093963 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:11.093963 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:11.093963 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:11.093963 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:11.093963 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:11.093963 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 08:28:11.093963 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:28:11.093963 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 08:28:11.093963 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:11.094968 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b212ad15-cbf6-4abf-817a-ee469b3089a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:11.094968 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b212ad15-cbf6-4abf-817a-ee469b3089a1 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:11.095102 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 389/766] 10.222.0.13 () {56 vars in 1129 bytes} [Fri Jun 24 08:28:10 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:11.095102 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df801d-e086-466a-a8b6-8c33e43b8557 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:11.095221 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df801d-e086-466a-a8b6-8c33e43b8557 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:11.096012 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df801d-e086-466a-a8b6-8c33e43b8557 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:11.097454 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 390/767] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:11.099138 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df801d-e086-466a-a8b6-8c33e43b8557 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:11.111530 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-b212ad15-cbf6-4abf-817a-ee469b3089a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:28:11.111530 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:11.111530 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:11.111530 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:11.111530 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:11.111530 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:11.111530 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:11.111530 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:11.111530 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:11.111530 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:11.111530 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:11.111530 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:11.111530 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:28:11.111530 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:28:11.111530 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:28:11.112608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:28:11.112608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:11.112608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:11.112608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:11.112608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:11.112608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:11.112608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:11.112608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:11.112608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:11.112608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:11.112608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:11.112608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:11.112608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:11.112608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:11.112608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:11.112608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:11.112608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:11.113671 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:28:11.113671 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:28:11.113671 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:28:11.113671 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:28:11.113671 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:28:11.113671 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:28:11.113671 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:28:11.113671 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:11.115951 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 378/768] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:28:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:28:11.117416 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade06465-b297-43ba-bc6d-db8c5ebc27ff None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:11.118347 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade06465-b297-43ba-bc6d-db8c5ebc27ff None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:11.119095 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade06465-b297-43ba-bc6d-db8c5ebc27ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:11.146486 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-ade06465-b297-43ba-bc6d-db8c5ebc27ff None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:11.147686 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-ade06465-b297-43ba-bc6d-db8c5ebc27ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:11.209117 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:11.238904 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 379/769] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:28:11.241300 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade06465-b297-43ba-bc6d-db8c5ebc27ff None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:11.245211 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df801d-e086-466a-a8b6-8c33e43b8557 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:11.304910 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769d08d3-8b5f-46b8-a0f6-0143ef40256d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:11.306206 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d08d3-8b5f-46b8-a0f6-0143ef40256d None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:11.306729 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d08d3-8b5f-46b8-a0f6-0143ef40256d None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:11.306729 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d08d3-8b5f-46b8-a0f6-0143ef40256d None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:11.307432 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769d08d3-8b5f-46b8-a0f6-0143ef40256d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:11.307773 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769d08d3-8b5f-46b8-a0f6-0143ef40256d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:11.309849 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769d08d3-8b5f-46b8-a0f6-0143ef40256d None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:11.317163 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 391/770] 10.222.0.13 () {56 vars in 1124 bytes} [Fri Jun 24 08:28:11 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:11.318200 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769d08d3-8b5f-46b8-a0f6-0143ef40256d None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:11.333193 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:11.391610 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade06465-b297-43ba-bc6d-db8c5ebc27ff None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:11.440366 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d866088-06e7-4a5a-9f6c-ae625215b8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:11.441311 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d866088-06e7-4a5a-9f6c-ae625215b8ab None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:11.442034 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d866088-06e7-4a5a-9f6c-ae625215b8ab None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:11.442468 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d866088-06e7-4a5a-9f6c-ae625215b8ab None admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:11.443683 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d866088-06e7-4a5a-9f6c-ae625215b8ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:11.444145 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d866088-06e7-4a5a-9f6c-ae625215b8ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:11.446443 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d866088-06e7-4a5a-9f6c-ae625215b8ab None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:11.454982 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769d08d3-8b5f-46b8-a0f6-0143ef40256d None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:11.468195 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 380/771] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Jun 24 08:28:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:11.468924 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d866088-06e7-4a5a-9f6c-ae625215b8ab None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:11.502689 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12d7e5-751a-46e7-ad79-78693a4f952f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:11.503705 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12d7e5-751a-46e7-ad79-78693a4f952f None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:11.504178 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12d7e5-751a-46e7-ad79-78693a4f952f None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:11.504720 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12d7e5-751a-46e7-ad79-78693a4f952f None admin] PATH_INFO: `/v3/services/network` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:11.505328 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c12d7e5-751a-46e7-ad79-78693a4f952f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:11.506822 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c12d7e5-751a-46e7-ad79-78693a4f952f None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:11.520159 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-6c12d7e5-751a-46e7-ad79-78693a4f952f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 08:28:11.520159 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:11.520159 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 08:28:11.520159 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 08:28:11.520159 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 08:28:11.520159 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 08:28:11.520159 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 08:28:11.520159 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:28:11.520159 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 08:28:11.520159 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:11.520159 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:28:11.520159 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:11.520159 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:11.520159 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:11.520159 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:11.520159 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:11.520159 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:11.521375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:11.521375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:11.521375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:11.521375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:11.521375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:11.521375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:11.521375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 08:28:11.521375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 08:28:11.521375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 08:28:11.521375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 08:28:11.521375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:11.521375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:11.521375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:11.521375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:11.521375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:11.521375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:11.522447 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:11.522447 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:11.522447 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:11.522447 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:11.522447 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:11.522447 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:11.522447 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:11.522447 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:11.522447 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:11.522447 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:11.522447 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 08:28:11.522447 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:28:11.522447 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 08:28:11.522447 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:11.529787 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 392/772] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:28:11 2022] GET /identity/v3/services/network => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:11.530721 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12d7e5-751a-46e7-ad79-78693a4f952f None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:11.535868 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:11.609256 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d866088-06e7-4a5a-9f6c-ae625215b8ab None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:11.640705 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:11.655688 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12d7e5-751a-46e7-ad79-78693a4f952f None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:11.656850 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd64a92-f48a-4738-a583-c55ec604ec05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:11.657999 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd64a92-f48a-4738-a583-c55ec604ec05 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:11.659395 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd64a92-f48a-4738-a583-c55ec604ec05 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:11.659868 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd64a92-f48a-4738-a583-c55ec604ec05 None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:11.660789 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd64a92-f48a-4738-a583-c55ec604ec05 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:11.661265 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd64a92-f48a-4738-a583-c55ec604ec05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:11.662917 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd64a92-f48a-4738-a583-c55ec604ec05 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:11.674884 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 381/773] 10.222.0.13 () {56 vars in 1124 bytes} [Fri Jun 24 08:28:11 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:11.676074 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd64a92-f48a-4738-a583-c55ec604ec05 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:11.703092 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da17dd75-9cbd-4ed0-b585-c072ceae7876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:11.703962 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da17dd75-9cbd-4ed0-b585-c072ceae7876 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:11.704377 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da17dd75-9cbd-4ed0-b585-c072ceae7876 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:11.704752 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da17dd75-9cbd-4ed0-b585-c072ceae7876 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:11.705806 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da17dd75-9cbd-4ed0-b585-c072ceae7876 None admin] RBAC: Adding query filter params (role.id=acfe707eee6944ad94d1915fc9ee7ea3, scope.project.id=180b0af5863349e99ae412b07c1e9465, user.id=bec8c2f113cf41a0939b81f38fbee813) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:11.706220 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da17dd75-9cbd-4ed0-b585-c072ceae7876 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:11.707815 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da17dd75-9cbd-4ed0-b585-c072ceae7876 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:11.726117 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 393/774] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Jun 24 08:28:11 2022] GET /identity/v3/role_assignments?user.id=bec8c2f113cf41a0939b81f38fbee813&scope.project.id=180b0af5863349e99ae412b07c1e9465&role.id=acfe707eee6944ad94d1915fc9ee7ea3 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:11.728212 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da17dd75-9cbd-4ed0-b585-c072ceae7876 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:11.855282 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:11.905058 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da17dd75-9cbd-4ed0-b585-c072ceae7876 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:11.955106 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b81d52-a954-41d3-b48b-ac8bf3cbbf51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:11.955772 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b81d52-a954-41d3-b48b-ac8bf3cbbf51 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:11.956065 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b81d52-a954-41d3-b48b-ac8bf3cbbf51 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:11.956345 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b81d52-a954-41d3-b48b-ac8bf3cbbf51 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:11.957144 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b81d52-a954-41d3-b48b-ac8bf3cbbf51 None admin] RBAC: Adding query filter params (interface=public, service_id=5219640313144502b850ae74173112bb, region_id=RegionOne) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:11.957457 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b81d52-a954-41d3-b48b-ac8bf3cbbf51 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:11.959141 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b81d52-a954-41d3-b48b-ac8bf3cbbf51 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:11.966635 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd64a92-f48a-4738-a583-c55ec604ec05 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:11.979019 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 394/775] 10.222.0.13 () {56 vars in 1251 bytes} [Fri Jun 24 08:28:11 2022] GET /identity/v3/endpoints?service_id=5219640313144502b850ae74173112bb&interface=public®ion_id=RegionOne => generated 194 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:11.987363 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b81d52-a954-41d3-b48b-ac8bf3cbbf51 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:12.047415 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d363c85-3369-401e-9272-d2164a4c56d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:12.047645 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d363c85-3369-401e-9272-d2164a4c56d5 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:12.047876 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d363c85-3369-401e-9272-d2164a4c56d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:12.048211 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d363c85-3369-401e-9272-d2164a4c56d5 None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:12.048873 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d363c85-3369-401e-9272-d2164a4c56d5 None admin] RBAC: Adding query filter params (name=network) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:12.049181 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d363c85-3369-401e-9272-d2164a4c56d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:12.052817 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d363c85-3369-401e-9272-d2164a4c56d5 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:12.058343 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 382/776] 10.222.0.13 () {56 vars in 1112 bytes} [Fri Jun 24 08:28:11 2022] GET /identity/v3/services?name=network => generated 124 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:12.066465 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d363c85-3369-401e-9272-d2164a4c56d5 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:12.143534 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b81d52-a954-41d3-b48b-ac8bf3cbbf51 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:12.178984 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:12.183078 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:12.194847 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b33d5-1965-4227-a8cb-4332f880ed1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:12.195955 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b33d5-1965-4227-a8cb-4332f880ed1e None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:12.197441 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b33d5-1965-4227-a8cb-4332f880ed1e None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:12.198047 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b33d5-1965-4227-a8cb-4332f880ed1e None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:12.201367 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093b33d5-1965-4227-a8cb-4332f880ed1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:12.204419 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093b33d5-1965-4227-a8cb-4332f880ed1e None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:12.216834 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 395/777] 10.222.0.13 () {56 vars in 1087 bytes} [Fri Jun 24 08:28:11 2022] GET /identity/v3/services => generated 772 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:12.264512 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d363c85-3369-401e-9272-d2164a4c56d5 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:12.304545 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db255ff6-37cd-4048-9e85-27857576f48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:12.305719 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db255ff6-37cd-4048-9e85-27857576f48f None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:12.306214 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db255ff6-37cd-4048-9e85-27857576f48f None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:12.306785 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db255ff6-37cd-4048-9e85-27857576f48f None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:12.307699 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db255ff6-37cd-4048-9e85-27857576f48f None admin] RBAC: Adding query filter params (type=network) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:12.308303 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db255ff6-37cd-4048-9e85-27857576f48f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:12.309901 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db255ff6-37cd-4048-9e85-27857576f48f None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:12.315647 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 383/778] 10.222.0.13 () {56 vars in 1112 bytes} [Fri Jun 24 08:28:12 2022] GET /identity/v3/services?type=network => generated 350 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:12.322200 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b33d5-1965-4227-a8cb-4332f880ed1e None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:12.439775 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b33d5-1965-4227-a8cb-4332f880ed1e None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:12.479214 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc8d53d-eb46-4a37-88bc-1a3c30adec6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:12.480355 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8d53d-eb46-4a37-88bc-1a3c30adec6e None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:12.480883 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8d53d-eb46-4a37-88bc-1a3c30adec6e None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:12.481404 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8d53d-eb46-4a37-88bc-1a3c30adec6e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:12.482520 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc8d53d-eb46-4a37-88bc-1a3c30adec6e None admin] RBAC: Adding query filter params (interface=public, service_id=d6fea3707b5246369b0149708834d17e, region_id=RegionOne) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:12.483056 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc8d53d-eb46-4a37-88bc-1a3c30adec6e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:12.484689 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc8d53d-eb46-4a37-88bc-1a3c30adec6e None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:12.491962 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 396/779] 10.222.0.13 () {56 vars in 1251 bytes} [Fri Jun 24 08:28:12 2022] GET /identity/v3/endpoints?service_id=d6fea3707b5246369b0149708834d17e&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:12.498444 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db255ff6-37cd-4048-9e85-27857576f48f None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:12.626448 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db255ff6-37cd-4048-9e85-27857576f48f None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:12.644505 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:12.672405 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff155901-1a1f-4184-b9c3-77e78e50764e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:12.673446 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff155901-1a1f-4184-b9c3-77e78e50764e None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:12.673969 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff155901-1a1f-4184-b9c3-77e78e50764e None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:12.674468 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff155901-1a1f-4184-b9c3-77e78e50764e None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:12.675410 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff155901-1a1f-4184-b9c3-77e78e50764e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:12.677178 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff155901-1a1f-4184-b9c3-77e78e50764e None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:12.683762 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 384/780] 10.222.0.13 () {56 vars in 1087 bytes} [Fri Jun 24 08:28:12 2022] GET /identity/v3/services => generated 772 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:12.814560 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d09275-8b26-4365-b702-0bc7fdfab9fa None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:12.815200 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d09275-8b26-4365-b702-0bc7fdfab9fa None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:12.815685 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d09275-8b26-4365-b702-0bc7fdfab9fa None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:12.817787 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 397/781] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:12.823715 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6890a9-470a-4999-b265-5e545ca8cfb4 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:12.824276 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6890a9-470a-4999-b265-5e545ca8cfb4 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:12.824758 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6890a9-470a-4999-b265-5e545ca8cfb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:12.848365 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-4a6890a9-470a-4999-b265-5e545ca8cfb4 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:12.849977 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-4a6890a9-470a-4999-b265-5e545ca8cfb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:12.934484 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 385/782] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:28:12.968141 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d09275-8b26-4365-b702-0bc7fdfab9fa None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:13.037773 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:13.094633 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d09275-8b26-4365-b702-0bc7fdfab9fa None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:13.094957 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2acdda-2f46-4c1d-b5ea-33572d565b21 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:13.095775 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2acdda-2f46-4c1d-b5ea-33572d565b21 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:13.096358 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2acdda-2f46-4c1d-b5ea-33572d565b21 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:13.098402 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 386/783] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:13.104549 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae2b87b-f0e8-4599-adae-8360f73979da None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:13.104845 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae2b87b-f0e8-4599-adae-8360f73979da None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:13.105106 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae2b87b-f0e8-4599-adae-8360f73979da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:13.141475 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-dae2b87b-f0e8-4599-adae-8360f73979da None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:13.143143 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-dae2b87b-f0e8-4599-adae-8360f73979da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:13.163793 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e57ff5-0307-4ab0-b8c4-e55f86771c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:13.164631 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e57ff5-0307-4ab0-b8c4-e55f86771c0c None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:13.165357 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e57ff5-0307-4ab0-b8c4-e55f86771c0c None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:13.165357 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e57ff5-0307-4ab0-b8c4-e55f86771c0c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:13.193952 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e57ff5-0307-4ab0-b8c4-e55f86771c0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:13.208646 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-f8e57ff5-0307-4ab0-b8c4-e55f86771c0c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:28:13.208646 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:13.208646 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:13.208646 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:13.208646 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:13.208646 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:13.208646 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:13.208646 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:13.208646 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:13.208646 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:13.208646 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:13.208646 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:13.208646 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:13.208646 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:13.208646 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:13.208646 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:13.209646 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:13.209646 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:13.209646 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:13.209646 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:13.209646 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:13.209646 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:13.209646 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:13.209646 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:13.209646 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:13.209646 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:13.209646 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:13.209646 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:13.209646 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:13.209646 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:13.209646 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:13.209646 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:13.210539 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:13.210539 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:13.210539 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:13.210539 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:13.210539 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:13.210539 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:13.210539 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:13.210539 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:13.210539 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:13.210539 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:13.210539 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:28:13.210539 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:13.210539 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 398/784] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Jun 24 08:28:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:13.218651 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e57ff5-0307-4ab0-b8c4-e55f86771c0c None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:13.239493 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 387/785] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 08:28:13.287137 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae2b87b-f0e8-4599-adae-8360f73979da None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:13.363700 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:13.408030 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e57ff5-0307-4ab0-b8c4-e55f86771c0c None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:13.440466 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae2b87b-f0e8-4599-adae-8360f73979da None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:13.458062 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:13.485383 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348051a0-1087-4d3e-baa8-39bfb08350a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:13.486078 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348051a0-1087-4d3e-baa8-39bfb08350a8 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:13.486365 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348051a0-1087-4d3e-baa8-39bfb08350a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:13.486643 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348051a0-1087-4d3e-baa8-39bfb08350a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:13.487397 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348051a0-1087-4d3e-baa8-39bfb08350a8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:13.487706 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348051a0-1087-4d3e-baa8-39bfb08350a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:13.489091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348051a0-1087-4d3e-baa8-39bfb08350a8 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:13.502216 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7da99a-659f-4a0d-bf31-3ed3405c2883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:13.503015 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7da99a-659f-4a0d-bf31-3ed3405c2883 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:13.503303 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7da99a-659f-4a0d-bf31-3ed3405c2883 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:13.503578 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7da99a-659f-4a0d-bf31-3ed3405c2883 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:13.509645 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 399/786] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:28:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:13.516035 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348051a0-1087-4d3e-baa8-39bfb08350a8 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:13.523164 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7da99a-659f-4a0d-bf31-3ed3405c2883 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:13.531563 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-ea7da99a-659f-4a0d-bf31-3ed3405c2883 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:28:13.531563 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:13.531563 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:13.531563 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:13.531563 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:13.531563 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:13.531563 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:13.531563 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:13.531563 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:13.531563 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:13.531563 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:13.531563 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:13.531563 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:13.531563 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:13.531563 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:13.531563 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:13.532479 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:13.532479 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:13.532479 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:13.532479 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:13.532479 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:13.532479 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:13.532479 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:13.532479 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:13.532479 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:13.532479 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:13.532479 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:13.532479 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:13.532479 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:13.532479 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:13.532479 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:13.532479 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:13.533285 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:13.533285 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:13.533285 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:13.533285 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:13.533285 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:13.533285 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:13.533285 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:13.533285 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:13.533285 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:13.533285 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:13.533285 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:28:13.533285 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:13.533285 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 388/787] 10.222.0.13 () {56 vars in 1102 bytes} [Fri Jun 24 08:28:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:28:13.541287 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7da99a-659f-4a0d-bf31-3ed3405c2883 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:13.692102 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348051a0-1087-4d3e-baa8-39bfb08350a8 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:13.731959 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:13.736479 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7da99a-659f-4a0d-bf31-3ed3405c2883 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:13.774773 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243e331-ac0e-4518-9875-e9eadc12fa71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:13.775916 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243e331-ac0e-4518-9875-e9eadc12fa71 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:13.776405 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243e331-ac0e-4518-9875-e9eadc12fa71 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:13.776878 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243e331-ac0e-4518-9875-e9eadc12fa71 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:13.777850 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8243e331-ac0e-4518-9875-e9eadc12fa71 None admin] RBAC: Adding query filter params (name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:13.778380 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8243e331-ac0e-4518-9875-e9eadc12fa71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:13.779980 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8243e331-ac0e-4518-9875-e9eadc12fa71 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:13.792808 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 389/788] 10.222.0.13 () {56 vars in 1103 bytes} [Fri Jun 24 08:28:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:13.795640 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243e331-ac0e-4518-9875-e9eadc12fa71 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:13.798961 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f89446-05f0-43af-a7f3-f1e4f679b174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:13.800543 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f89446-05f0-43af-a7f3-f1e4f679b174 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:13.800543 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f89446-05f0-43af-a7f3-f1e4f679b174 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:13.800543 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f89446-05f0-43af-a7f3-f1e4f679b174 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:13.818943 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f89446-05f0-43af-a7f3-f1e4f679b174 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:13.820531 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f89446-05f0-43af-a7f3-f1e4f679b174 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:13.862463 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-f1f89446-05f0-43af-a7f3-f1e4f679b174 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:13.862463 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:13.862463 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:13.862463 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:13.862463 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:13.862463 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:13.862463 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:13.862463 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:13.862463 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:13.862463 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:13.862463 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:13.862463 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:13.862463 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:28:13.862463 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:28:13.862463 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:28:13.863422 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:28:13.863422 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:13.863422 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:13.863422 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:13.863422 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:13.863422 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:13.863422 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:13.863422 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:13.863422 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:13.863422 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:13.863422 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:13.863422 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:13.863422 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:13.863422 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:13.863422 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:13.863422 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:13.863422 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:13.864897 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:28:13.864897 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:28:13.864897 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:13.864897 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:13.864897 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 400/789] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:13.870261 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f89446-05f0-43af-a7f3-f1e4f679b174 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:13.910901 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243e331-ac0e-4518-9875-e9eadc12fa71 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:13.938326 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:13.949601 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786b9387-f45f-4cac-820a-9556847ca6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:13.950745 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b9387-f45f-4cac-820a-9556847ca6bc None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:13.951228 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b9387-f45f-4cac-820a-9556847ca6bc None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:13.951814 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b9387-f45f-4cac-820a-9556847ca6bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:13.963175 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786b9387-f45f-4cac-820a-9556847ca6bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:13.964899 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786b9387-f45f-4cac-820a-9556847ca6bc None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:13.982382 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-786b9387-f45f-4cac-820a-9556847ca6bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:13.982382 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:13.982382 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:13.982382 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:13.982382 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:13.982382 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:13.982382 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:13.982382 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:13.982382 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:13.982382 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:13.982382 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:13.982382 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:13.982382 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:28:13.982382 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:28:13.982382 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:28:13.983312 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:28:13.983312 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:13.983312 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:13.983312 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:13.983312 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:13.983312 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:13.983312 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:13.983312 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:13.983312 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:13.983312 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:13.983312 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:13.983312 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:13.983312 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:13.983312 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:13.983312 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:13.983312 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:13.983312 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:13.984176 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:28:13.984176 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:28:13.984176 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:13.984176 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:13.990737 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 390/790] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:28:13.992158 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786b9387-f45f-4cac-820a-9556847ca6bc None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:14.026034 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:14.082438 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f89446-05f0-43af-a7f3-f1e4f679b174 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:14.128028 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786b9387-f45f-4cac-820a-9556847ca6bc None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:14.167832 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2b15ea-d5aa-4ea3-a94a-b72d26cec191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:14.168519 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b15ea-d5aa-4ea3-a94a-b72d26cec191 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:14.168808 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b15ea-d5aa-4ea3-a94a-b72d26cec191 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:14.169088 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b15ea-d5aa-4ea3-a94a-b72d26cec191 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:14.169921 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2b15ea-d5aa-4ea3-a94a-b72d26cec191 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:14.170231 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2b15ea-d5aa-4ea3-a94a-b72d26cec191 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:14.171678 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2b15ea-d5aa-4ea3-a94a-b72d26cec191 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:14.193325 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748885b6-0ecf-4313-9c6d-fc1cc1deb3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:14.194035 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748885b6-0ecf-4313-9c6d-fc1cc1deb3cb None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:14.194328 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748885b6-0ecf-4313-9c6d-fc1cc1deb3cb None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:14.194610 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748885b6-0ecf-4313-9c6d-fc1cc1deb3cb None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:14.195366 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 401/791] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:28:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:14.195481 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748885b6-0ecf-4313-9c6d-fc1cc1deb3cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:14.195747 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748885b6-0ecf-4313-9c6d-fc1cc1deb3cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:14.197458 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748885b6-0ecf-4313-9c6d-fc1cc1deb3cb None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:14.201571 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2b15ea-d5aa-4ea3-a94a-b72d26cec191 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:14.213828 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 391/792] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:28:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:14.216269 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748885b6-0ecf-4313-9c6d-fc1cc1deb3cb None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:14.307604 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:14.335005 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748885b6-0ecf-4313-9c6d-fc1cc1deb3cb None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:14.379729 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab5df7b-3120-490e-a93b-59e55d1e8916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:14.380817 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab5df7b-3120-490e-a93b-59e55d1e8916 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:14.381103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab5df7b-3120-490e-a93b-59e55d1e8916 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:14.381379 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab5df7b-3120-490e-a93b-59e55d1e8916 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:14.385641 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:14.392394 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab5df7b-3120-490e-a93b-59e55d1e8916 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:14.393878 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab5df7b-3120-490e-a93b-59e55d1e8916 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:14.394410 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2b15ea-d5aa-4ea3-a94a-b72d26cec191 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:14.412239 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-7ab5df7b-3120-490e-a93b-59e55d1e8916 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 08:28:14.412239 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:14.412239 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:14.412239 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:14.412239 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:14.412239 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:14.412239 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:14.412239 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:14.412239 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:14.412239 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:14.412239 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:14.412239 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:14.412239 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:28:14.412239 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:28:14.412239 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:28:14.412239 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:28:14.413438 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:14.413438 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:14.413438 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:28:14.413438 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:28:14.413438 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:28:14.413438 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:28:14.413438 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:14.413438 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:14.413438 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:14.413438 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:14.413438 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:14.413438 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:14.413438 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:14.413438 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:14.413438 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:14.413438 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:14.413438 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:14.413438 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:14.415083 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:14.415083 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:14.415083 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:28:14.415083 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:28:14.415083 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:28:14.415083 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:28:14.415083 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:28:14.415083 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:28:14.415083 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 08:28:14.415083 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:14.417797 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 392/793] 10.222.0.13 () {56 vars in 1117 bytes} [Fri Jun 24 08:28:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:28:14.419965 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27afb3c8-581d-41da-98bc-90cb8f844d4f None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:14.420258 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27afb3c8-581d-41da-98bc-90cb8f844d4f None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:14.420538 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27afb3c8-581d-41da-98bc-90cb8f844d4f None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:14.425863 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 393/794] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:14.426627 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27afb3c8-581d-41da-98bc-90cb8f844d4f None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:14.443268 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79637374-6f7a-4792-8d88-daf8e705d99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:14.444855 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79637374-6f7a-4792-8d88-daf8e705d99e None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:14.445350 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79637374-6f7a-4792-8d88-daf8e705d99e None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:14.446635 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79637374-6f7a-4792-8d88-daf8e705d99e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:14.456019 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79637374-6f7a-4792-8d88-daf8e705d99e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:14.458107 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79637374-6f7a-4792-8d88-daf8e705d99e None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:14.477538 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-79637374-6f7a-4792-8d88-daf8e705d99e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 08:28:14.477538 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:14.477538 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:14.477538 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:14.477538 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:14.477538 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:14.477538 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:14.477538 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:14.477538 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:14.477538 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:14.477538 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:14.477538 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:14.477538 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:28:14.477538 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:28:14.477538 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:28:14.477538 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:28:14.478553 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:14.478553 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:14.478553 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:28:14.478553 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:28:14.478553 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:28:14.478553 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:28:14.478553 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:14.478553 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:14.478553 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:14.478553 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:14.478553 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:14.478553 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:14.478553 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:14.478553 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:14.478553 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:14.478553 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:14.478553 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:14.478553 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:14.479484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:14.479484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:14.479484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:28:14.479484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:28:14.479484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:28:14.479484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:28:14.479484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:28:14.479484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:28:14.479484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 08:28:14.479484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:14.482378 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 402/795] 10.222.0.13 () {56 vars in 1093 bytes} [Fri Jun 24 08:28:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:28:14.483949 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db4b84-c6fd-417a-a26a-6a4b91d40a6d None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:14.484369 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db4b84-c6fd-417a-a26a-6a4b91d40a6d None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:14.484774 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db4b84-c6fd-417a-a26a-6a4b91d40a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:14.506654 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-04db4b84-c6fd-417a-a26a-6a4b91d40a6d None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:14.509271 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-04db4b84-c6fd-417a-a26a-6a4b91d40a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:14.546418 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27afb3c8-581d-41da-98bc-90cb8f844d4f None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:14.580453 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:14.588794 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10599ec-6983-4ea3-97d5-d7e9d56c48b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:14.589400 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10599ec-6983-4ea3-97d5-d7e9d56c48b0 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:14.589880 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10599ec-6983-4ea3-97d5-d7e9d56c48b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:14.590253 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10599ec-6983-4ea3-97d5-d7e9d56c48b0 None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:14.591228 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10599ec-6983-4ea3-97d5-d7e9d56c48b0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:14.591739 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10599ec-6983-4ea3-97d5-d7e9d56c48b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:14.593391 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10599ec-6983-4ea3-97d5-d7e9d56c48b0 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:14.598421 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 403/796] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:28:14.599566 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04db4b84-c6fd-417a-a26a-6a4b91d40a6d None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:14.630187 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 394/797] 10.222.0.13 () {56 vars in 1149 bytes} [Fri Jun 24 08:28:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:14.638585 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10599ec-6983-4ea3-97d5-d7e9d56c48b0 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:14.702967 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:14.722939 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04db4b84-c6fd-417a-a26a-6a4b91d40a6d None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:14.762903 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d70226-e212-43dd-b838-b4315076f496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:14.763730 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d70226-e212-43dd-b838-b4315076f496 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:14.764209 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d70226-e212-43dd-b838-b4315076f496 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:14.764582 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d70226-e212-43dd-b838-b4315076f496 None admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:14.765717 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d70226-e212-43dd-b838-b4315076f496 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:14.766218 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d70226-e212-43dd-b838-b4315076f496 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:14.767674 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d70226-e212-43dd-b838-b4315076f496 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:14.768571 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10599ec-6983-4ea3-97d5-d7e9d56c48b0 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:14.792399 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 404/798] 10.222.0.13 () {56 vars in 1133 bytes} [Fri Jun 24 08:28:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:14.793377 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d70226-e212-43dd-b838-b4315076f496 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:14.812854 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c55b761-8260-401d-a3c0-3ceff49fb6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:14.813946 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55b761-8260-401d-a3c0-3ceff49fb6b9 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:14.814332 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55b761-8260-401d-a3c0-3ceff49fb6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:14.814709 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55b761-8260-401d-a3c0-3ceff49fb6b9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:14.827061 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c55b761-8260-401d-a3c0-3ceff49fb6b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:14.837232 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-7c55b761-8260-401d-a3c0-3ceff49fb6b9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:28:14.837232 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:14.837232 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:14.837232 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:14.837232 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:14.837232 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:14.837232 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:14.837232 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:14.837232 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:14.837232 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:14.837232 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:14.837232 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:14.837232 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:14.837232 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:14.837232 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:14.837232 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:14.838764 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:14.838764 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:14.838764 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:14.838764 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:14.838764 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:14.838764 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:14.838764 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:14.838764 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:14.838764 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:14.838764 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:14.838764 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:14.838764 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:14.838764 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:14.838764 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:14.838764 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:14.838764 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:14.839878 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:14.839878 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:14.839878 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:14.839878 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:14.839878 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:14.839878 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:14.839878 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:14.839878 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:14.839878 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:14.839878 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:14.839878 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:28:14.839878 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:14.846364 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 395/799] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Jun 24 08:28:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:14.847362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c55b761-8260-401d-a3c0-3ceff49fb6b9 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:14.908231 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:14.913480 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d70226-e212-43dd-b838-b4315076f496 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:14.953325 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f0bdf3-c3a5-4cbe-8d5e-cf70557cf74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:14.954624 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f0bdf3-c3a5-4cbe-8d5e-cf70557cf74d None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:14.955223 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f0bdf3-c3a5-4cbe-8d5e-cf70557cf74d None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:14.955673 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f0bdf3-c3a5-4cbe-8d5e-cf70557cf74d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:14.964807 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c55b761-8260-401d-a3c0-3ceff49fb6b9 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:14.967993 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f0bdf3-c3a5-4cbe-8d5e-cf70557cf74d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:14.969607 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f0bdf3-c3a5-4cbe-8d5e-cf70557cf74d None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:14.971377 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:14.990568 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-48f0bdf3-c3a5-4cbe-8d5e-cf70557cf74d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:14.990568 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:14.990568 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:14.990568 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:14.990568 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:14.990568 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:14.990568 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:14.990568 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:14.990568 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:14.990568 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:14.990568 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:14.990568 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:14.990568 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:28:14.990568 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:28:14.990568 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:28:14.991633 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:28:14.991633 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:14.991633 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:14.991633 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:14.991633 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:14.991633 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:14.991633 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:14.991633 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:14.991633 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:14.991633 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:14.991633 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:14.991633 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:14.991633 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:14.991633 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:14.991633 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:14.991633 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:14.991633 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:14.992542 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:28:14.992542 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:28:14.992542 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:14.992542 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:14.992542 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 405/800] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:14.992542 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f0bdf3-c3a5-4cbe-8d5e-cf70557cf74d None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:15.007669 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b944d14-4720-4b10-a979-25a5ae02b09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:15.008469 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b944d14-4720-4b10-a979-25a5ae02b09f None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:15.008767 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b944d14-4720-4b10-a979-25a5ae02b09f None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:15.009048 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b944d14-4720-4b10-a979-25a5ae02b09f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:15.079763 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b944d14-4720-4b10-a979-25a5ae02b09f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:15.081229 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b944d14-4720-4b10-a979-25a5ae02b09f None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:15.103159 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f0bdf3-c3a5-4cbe-8d5e-cf70557cf74d None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:15.131485 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:15.139608 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6be20d-3847-4b47-aef0-95bf8a242f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:15.140271 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6be20d-3847-4b47-aef0-95bf8a242f15 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:15.140553 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6be20d-3847-4b47-aef0-95bf8a242f15 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:15.140824 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6be20d-3847-4b47-aef0-95bf8a242f15 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:15.141561 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6be20d-3847-4b47-aef0-95bf8a242f15 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:15.141891 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6be20d-3847-4b47-aef0-95bf8a242f15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:15.143262 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6be20d-3847-4b47-aef0-95bf8a242f15 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:15.151228 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 406/801] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:28:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:15.152425 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e008b40-ced9-4643-a626-bd3e84c17735 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:15.152712 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e008b40-ced9-4643-a626-bd3e84c17735 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:15.152988 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e008b40-ced9-4643-a626-bd3e84c17735 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:15.154226 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 407/802] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:15.154904 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e008b40-ced9-4643-a626-bd3e84c17735 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:15.187156 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 396/803] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:28:14 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 08:28:15.188538 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9a559-653c-4bb7-9157-19754cead4b9 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:15.188830 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9a559-653c-4bb7-9157-19754cead4b9 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:15.189105 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9a559-653c-4bb7-9157-19754cead4b9 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:15.190494 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 397/804] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:15.191038 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d9a559-653c-4bb7-9157-19754cead4b9 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:15.216693 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:15.262150 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e008b40-ced9-4643-a626-bd3e84c17735 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:15.293085 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d9a559-653c-4bb7-9157-19754cead4b9 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:15.295627 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb8962-98f0-4eaf-bca4-312e34d5afef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:15.296285 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb8962-98f0-4eaf-bca4-312e34d5afef None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:15.296567 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb8962-98f0-4eaf-bca4-312e34d5afef None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:15.296842 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb8962-98f0-4eaf-bca4-312e34d5afef None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:15.297550 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccb8962-98f0-4eaf-bca4-312e34d5afef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:15.297888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccb8962-98f0-4eaf-bca4-312e34d5afef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:15.299261 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccb8962-98f0-4eaf-bca4-312e34d5afef None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:15.312394 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 408/805] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:28:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:15.313591 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f3f31a-c380-4308-8a0b-1be2eb10f1fd None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:15.313973 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f3f31a-c380-4308-8a0b-1be2eb10f1fd None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:15.314249 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f3f31a-c380-4308-8a0b-1be2eb10f1fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:15.329915 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-93f3f31a-c380-4308-8a0b-1be2eb10f1fd None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:15.331108 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-93f3f31a-c380-4308-8a0b-1be2eb10f1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:15.331707 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a02400-1392-49a1-a036-d9f1c0a72c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:15.332485 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a02400-1392-49a1-a036-d9f1c0a72c7b None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:15.332766 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a02400-1392-49a1-a036-d9f1c0a72c7b None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:15.333037 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a02400-1392-49a1-a036-d9f1c0a72c7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:15.343902 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a02400-1392-49a1-a036-d9f1c0a72c7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:15.345331 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a02400-1392-49a1-a036-d9f1c0a72c7b None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:15.363412 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-62a02400-1392-49a1-a036-d9f1c0a72c7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:15.363412 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:15.363412 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:15.363412 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:15.363412 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:15.363412 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:15.363412 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:15.363412 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:15.363412 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:15.363412 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:15.363412 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:15.363412 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:15.363412 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:28:15.363412 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:28:15.363412 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:28:15.364484 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:28:15.364484 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:15.364484 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:15.364484 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:15.364484 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:15.364484 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:15.364484 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:15.364484 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:15.364484 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:15.364484 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:15.364484 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:15.364484 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:15.364484 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:15.364484 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:15.364484 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:15.364484 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:15.364484 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:15.365565 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:28:15.365565 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:28:15.365565 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:15.365565 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:15.365565 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 398/806] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:15.365565 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a02400-1392-49a1-a036-d9f1c0a72c7b None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:15.370442 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:15.407490 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 409/807] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:28:15.408860 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c9210-3b52-439a-b8d7-c73cdc8d2790 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:15.409158 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c9210-3b52-439a-b8d7-c73cdc8d2790 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:15.409440 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c9210-3b52-439a-b8d7-c73cdc8d2790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:15.428466 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-966c9210-3b52-439a-b8d7-c73cdc8d2790 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:15.429837 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-966c9210-3b52-439a-b8d7-c73cdc8d2790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:15.473029 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:15.475824 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a02400-1392-49a1-a036-d9f1c0a72c7b None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:15.498396 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 410/808] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:28:15.499206 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966c9210-3b52-439a-b8d7-c73cdc8d2790 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:15.516510 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457f2be7-3cad-4ddd-8d0c-e015f7f6f798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:15.517237 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457f2be7-3cad-4ddd-8d0c-e015f7f6f798 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:15.517517 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457f2be7-3cad-4ddd-8d0c-e015f7f6f798 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:15.517819 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457f2be7-3cad-4ddd-8d0c-e015f7f6f798 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:15.529206 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457f2be7-3cad-4ddd-8d0c-e015f7f6f798 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:15.530669 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457f2be7-3cad-4ddd-8d0c-e015f7f6f798 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:15.547649 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-457f2be7-3cad-4ddd-8d0c-e015f7f6f798 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:15.547649 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:15.547649 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:15.547649 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:15.547649 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:15.547649 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:15.547649 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:15.547649 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:15.547649 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:15.547649 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:15.547649 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:15.547649 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:15.547649 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:28:15.547649 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:28:15.547649 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:28:15.548542 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:28:15.548542 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:15.548542 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:15.548542 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:15.548542 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:15.548542 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:15.548542 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:15.548542 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:15.548542 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:15.548542 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:15.548542 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:15.548542 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:15.548542 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:15.548542 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:15.548542 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:15.548542 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:15.548542 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:15.549405 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:28:15.549405 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:28:15.549405 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:15.549405 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:15.549405 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 399/809] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:15.549683 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457f2be7-3cad-4ddd-8d0c-e015f7f6f798 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:15.603797 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966c9210-3b52-439a-b8d7-c73cdc8d2790 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:15.604685 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:15.638140 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920cced8-634b-4ab5-b4c0-3d85d37f1104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:15.638931 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920cced8-634b-4ab5-b4c0-3d85d37f1104 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:15.639214 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920cced8-634b-4ab5-b4c0-3d85d37f1104 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:15.639489 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920cced8-634b-4ab5-b4c0-3d85d37f1104 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:15.646104 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920cced8-634b-4ab5-b4c0-3d85d37f1104 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:15.647537 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920cced8-634b-4ab5-b4c0-3d85d37f1104 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:15.650747 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457f2be7-3cad-4ddd-8d0c-e015f7f6f798 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:15.661532 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-920cced8-634b-4ab5-b4c0-3d85d37f1104 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:28:15.661532 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:15.661532 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:15.661532 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:15.661532 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:15.661532 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:15.661532 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:15.661532 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:15.661532 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:15.661532 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:15.661532 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:15.661532 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:15.661532 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:28:15.661532 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:28:15.661532 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:28:15.662604 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:28:15.662604 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:15.662604 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:15.662604 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:15.662604 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:15.662604 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:15.662604 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:15.662604 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:15.662604 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:15.662604 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:15.662604 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:15.662604 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:15.662604 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:15.662604 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:15.662604 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:15.662604 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:15.662604 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:15.663616 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:28:15.663616 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:28:15.663616 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:28:15.663616 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:28:15.663616 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:28:15.663616 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:28:15.663616 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:28:15.663616 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:15.663616 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 411/810] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:28:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:15.663616 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920cced8-634b-4ab5-b4c0-3d85d37f1104 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:15.689643 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e36cb7f-0d56-4b8e-a5f5-412788e865de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:15.690329 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e36cb7f-0d56-4b8e-a5f5-412788e865de None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:15.690618 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e36cb7f-0d56-4b8e-a5f5-412788e865de None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:15.690898 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e36cb7f-0d56-4b8e-a5f5-412788e865de None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:15.691614 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e36cb7f-0d56-4b8e-a5f5-412788e865de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:15.691925 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e36cb7f-0d56-4b8e-a5f5-412788e865de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:15.693289 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e36cb7f-0d56-4b8e-a5f5-412788e865de None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:15.700623 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:15.705993 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 400/811] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:28:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:15.706883 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e36cb7f-0d56-4b8e-a5f5-412788e865de None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:15.778486 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920cced8-634b-4ab5-b4c0-3d85d37f1104 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:15.806901 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e36cb7f-0d56-4b8e-a5f5-412788e865de None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:15.811904 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770d327b-7dff-4f30-8f4b-894e3304d883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:15.812717 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d327b-7dff-4f30-8f4b-894e3304d883 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:15.813006 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d327b-7dff-4f30-8f4b-894e3304d883 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:15.813296 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d327b-7dff-4f30-8f4b-894e3304d883 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:15.813982 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770d327b-7dff-4f30-8f4b-894e3304d883 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:15.815357 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770d327b-7dff-4f30-8f4b-894e3304d883 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:15.820301 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:15.828246 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-770d327b-7dff-4f30-8f4b-894e3304d883 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 08:28:15.828246 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:15.828246 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 08:28:15.828246 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 08:28:15.828246 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 08:28:15.828246 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 08:28:15.828246 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 08:28:15.828246 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:28:15.828246 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 08:28:15.828246 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:15.828246 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:28:15.828246 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:15.828246 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:15.828246 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:15.828246 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:15.828246 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:15.829208 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:15.829208 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:15.829208 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:15.829208 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:15.829208 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:15.829208 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:15.829208 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:15.829208 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 08:28:15.829208 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 08:28:15.829208 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 08:28:15.829208 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 08:28:15.829208 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:15.829208 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:15.829208 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:15.829208 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:15.829208 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:15.830074 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:15.830074 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:15.830074 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:15.830074 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:15.830074 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:15.830074 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:15.830074 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:15.830074 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:15.830074 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:15.830074 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:15.830074 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:15.830074 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 08:28:15.830074 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:28:15.830074 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 08:28:15.830074 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:15.830074 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 412/812] 10.222.0.13 () {56 vars in 1129 bytes} [Fri Jun 24 08:28:15 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:15.831084 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770d327b-7dff-4f30-8f4b-894e3304d883 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:15.847214 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caa498d-2248-4e6a-9294-d60f0a815404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:15.847998 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa498d-2248-4e6a-9294-d60f0a815404 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:15.848278 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa498d-2248-4e6a-9294-d60f0a815404 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:15.848545 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa498d-2248-4e6a-9294-d60f0a815404 None admin] PATH_INFO: `/v3/services/network` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:15.849181 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caa498d-2248-4e6a-9294-d60f0a815404 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:15.850568 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caa498d-2248-4e6a-9294-d60f0a815404 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:15.863720 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-9caa498d-2248-4e6a-9294-d60f0a815404 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 08:28:15.863720 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:15.863720 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 08:28:15.863720 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 08:28:15.863720 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 08:28:15.863720 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 08:28:15.863720 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 08:28:15.863720 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:28:15.863720 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 08:28:15.863720 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:15.863720 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:28:15.863720 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:15.863720 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:15.863720 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:15.863720 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:15.863720 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:15.863720 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:15.869829 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:15.869829 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:15.869829 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:15.869829 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:15.869829 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:15.869829 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:15.869829 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 08:28:15.869829 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 08:28:15.869829 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 08:28:15.869829 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 08:28:15.869829 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:15.869829 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:15.869829 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:15.869829 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:15.869829 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:15.869829 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:15.870910 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:15.870910 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:15.870910 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:15.870910 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:15.870910 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:15.870910 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:15.870910 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:15.870910 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:15.870910 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:15.870910 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:15.870910 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 08:28:15.870910 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:28:15.870910 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 08:28:15.870910 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:15.870910 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 401/813] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:28:15 2022] GET /identity/v3/services/network => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:15.871800 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caa498d-2248-4e6a-9294-d60f0a815404 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:15.928663 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:15.932045 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770d327b-7dff-4f30-8f4b-894e3304d883 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:15.969143 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfb2368-affc-4c23-9d4a-cfa3e4596c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:15.969347 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caa498d-2248-4e6a-9294-d60f0a815404 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:15.969845 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb2368-affc-4c23-9d4a-cfa3e4596c02 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:15.970138 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb2368-affc-4c23-9d4a-cfa3e4596c02 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:15.970418 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb2368-affc-4c23-9d4a-cfa3e4596c02 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:15.971142 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfb2368-affc-4c23-9d4a-cfa3e4596c02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:15.971451 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfb2368-affc-4c23-9d4a-cfa3e4596c02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:15.972826 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfb2368-affc-4c23-9d4a-cfa3e4596c02 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:15.984568 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 413/814] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:28:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:15.985343 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfb2368-affc-4c23-9d4a-cfa3e4596c02 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:16.003490 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115446fb-2c74-4ecf-89c2-8677f0d66054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:16.004150 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115446fb-2c74-4ecf-89c2-8677f0d66054 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:16.004435 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115446fb-2c74-4ecf-89c2-8677f0d66054 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:16.004715 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115446fb-2c74-4ecf-89c2-8677f0d66054 None admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:16.005487 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115446fb-2c74-4ecf-89c2-8677f0d66054 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:16.005826 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115446fb-2c74-4ecf-89c2-8677f0d66054 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:16.007252 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115446fb-2c74-4ecf-89c2-8677f0d66054 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:16.020774 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 402/815] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Jun 24 08:28:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:16.021759 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115446fb-2c74-4ecf-89c2-8677f0d66054 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:16.079988 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:16.123179 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfb2368-affc-4c23-9d4a-cfa3e4596c02 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:16.143251 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115446fb-2c74-4ecf-89c2-8677f0d66054 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:16.180668 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb7f5e6-82f8-4b11-b24d-165d90310275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:16.182056 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7f5e6-82f8-4b11-b24d-165d90310275 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:16.182343 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7f5e6-82f8-4b11-b24d-165d90310275 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:16.182619 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7f5e6-82f8-4b11-b24d-165d90310275 None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:16.183342 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb7f5e6-82f8-4b11-b24d-165d90310275 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:16.183653 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb7f5e6-82f8-4b11-b24d-165d90310275 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:16.185071 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb7f5e6-82f8-4b11-b24d-165d90310275 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:16.190334 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 403/816] 10.222.0.13 () {56 vars in 1124 bytes} [Fri Jun 24 08:28:16 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:16.191181 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb7f5e6-82f8-4b11-b24d-165d90310275 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:16.273876 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da694429-a838-4e56-b254-9dfc879db1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:16.274961 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da694429-a838-4e56-b254-9dfc879db1ca None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:16.275238 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da694429-a838-4e56-b254-9dfc879db1ca None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:16.275508 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da694429-a838-4e56-b254-9dfc879db1ca None admin] PATH_INFO: `/v3/users/swift` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:16.282725 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da694429-a838-4e56-b254-9dfc879db1ca None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:16.284141 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da694429-a838-4e56-b254-9dfc879db1ca None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:16.296821 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb7f5e6-82f8-4b11-b24d-165d90310275 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:16.298271 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-da694429-a838-4e56-b254-9dfc879db1ca None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 08:28:16.298271 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:16.298271 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:16.298271 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:16.298271 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:16.298271 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:16.298271 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:16.298271 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:16.298271 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:16.298271 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:16.298271 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:16.298271 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:16.298271 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:28:16.298271 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:28:16.298271 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:28:16.298271 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:28:16.299210 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:16.299210 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:16.299210 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:28:16.299210 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:28:16.299210 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:28:16.299210 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:28:16.299210 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:16.299210 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:16.299210 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:16.299210 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:16.299210 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:16.299210 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:16.299210 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:16.299210 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:16.299210 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:16.299210 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:16.299210 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:16.299210 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:16.300165 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:16.300165 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:16.300165 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:28:16.300165 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:28:16.300165 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:28:16.300165 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:28:16.300165 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:28:16.300165 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:28:16.300165 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 08:28:16.300165 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:16.300693 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:16.300763 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 414/817] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Jun 24 08:28:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:16.300763 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da694429-a838-4e56-b254-9dfc879db1ca None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:16.323908 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:16.335816 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575459f8-4f60-4771-94f7-67dcc0151e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:16.336423 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575459f8-4f60-4771-94f7-67dcc0151e1b None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:16.336826 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575459f8-4f60-4771-94f7-67dcc0151e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:16.337242 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575459f8-4f60-4771-94f7-67dcc0151e1b None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:16.338142 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575459f8-4f60-4771-94f7-67dcc0151e1b None admin] RBAC: Adding query filter params (name=network) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:16.338451 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575459f8-4f60-4771-94f7-67dcc0151e1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:16.339825 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575459f8-4f60-4771-94f7-67dcc0151e1b None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:16.344612 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 404/818] 10.222.0.13 () {56 vars in 1112 bytes} [Fri Jun 24 08:28:16 2022] GET /identity/v3/services?name=network => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:16.345253 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575459f8-4f60-4771-94f7-67dcc0151e1b None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:16.413430 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da694429-a838-4e56-b254-9dfc879db1ca None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:16.448677 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec10e28-eea5-495a-b2a4-8793941a4152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:16.449447 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575459f8-4f60-4771-94f7-67dcc0151e1b None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:16.449554 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec10e28-eea5-495a-b2a4-8793941a4152 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:16.449752 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec10e28-eea5-495a-b2a4-8793941a4152 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:16.450025 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec10e28-eea5-495a-b2a4-8793941a4152 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:16.456590 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec10e28-eea5-495a-b2a4-8793941a4152 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:16.458052 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec10e28-eea5-495a-b2a4-8793941a4152 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:16.471357 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-cec10e28-eea5-495a-b2a4-8793941a4152 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:28:16.471357 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:16.471357 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:16.471357 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:16.471357 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:16.471357 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:16.471357 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:16.471357 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:16.471357 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:16.471357 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:16.471357 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:16.471357 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:16.471357 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:28:16.471357 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:28:16.471357 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:28:16.472279 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:28:16.472279 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:16.472279 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:16.472279 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:16.472279 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:16.472279 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:16.472279 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:16.472279 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:16.472279 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:16.472279 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:16.472279 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:16.472279 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:16.472279 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:16.472279 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:16.472279 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:16.472279 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:16.472279 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:16.473172 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:28:16.473172 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:28:16.473172 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:28:16.473172 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:28:16.473172 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:28:16.473172 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:28:16.473172 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:28:16.473172 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:16.473172 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 415/819] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:28:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:16.473906 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec10e28-eea5-495a-b2a4-8793941a4152 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:16.487038 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54f208b-e1da-4e69-9167-252178d44688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:16.487913 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f208b-e1da-4e69-9167-252178d44688 None admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:16.488195 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f208b-e1da-4e69-9167-252178d44688 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:16.488469 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f208b-e1da-4e69-9167-252178d44688 None admin] PATH_INFO: `/v3/projects/180b0af5863349e99ae412b07c1e9465/users/85d9c4c57681447c898ebfd7990b05b6/roles/6ca1ae4292d9414ca6526c6e8608de9e` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:16.503471 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54f208b-e1da-4e69-9167-252178d44688 None admin] RBAC: Authorizing `identity:create_grant(project_id=180b0af5863349e99ae412b07c1e9465, user_id=85d9c4c57681447c898ebfd7990b05b6, role_id=6ca1ae4292d9414ca6526c6e8608de9e)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:16.504891 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54f208b-e1da-4e69-9167-252178d44688 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:16.517198 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 405/820] 10.222.0.13 () {58 vars in 1439 bytes} [Fri Jun 24 08:28:16 2022] PUT /identity/v3/projects/180b0af5863349e99ae412b07c1e9465/users/85d9c4c57681447c898ebfd7990b05b6/roles/6ca1ae4292d9414ca6526c6e8608de9e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:28:16.518025 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54f208b-e1da-4e69-9167-252178d44688 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:16.547447 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:16.574930 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:16.598283 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec10e28-eea5-495a-b2a4-8793941a4152 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:16.636347 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a860eb15-627a-46bc-a544-30fe8eb4f044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:16.637027 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a860eb15-627a-46bc-a544-30fe8eb4f044 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:16.637315 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a860eb15-627a-46bc-a544-30fe8eb4f044 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:16.637593 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a860eb15-627a-46bc-a544-30fe8eb4f044 None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:16.638359 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a860eb15-627a-46bc-a544-30fe8eb4f044 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:16.638669 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a860eb15-627a-46bc-a544-30fe8eb4f044 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:16.640068 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a860eb15-627a-46bc-a544-30fe8eb4f044 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:16.645439 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54f208b-e1da-4e69-9167-252178d44688 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:16.650451 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 416/821] 10.222.0.13 () {56 vars in 1124 bytes} [Fri Jun 24 08:28:16 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:16.651429 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a860eb15-627a-46bc-a544-30fe8eb4f044 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:16.687905 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d815f54-0bb9-49ee-abed-e26f295154d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:16.689145 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d815f54-0bb9-49ee-abed-e26f295154d8 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:16.689430 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d815f54-0bb9-49ee-abed-e26f295154d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:16.689710 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d815f54-0bb9-49ee-abed-e26f295154d8 None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:16.690575 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d815f54-0bb9-49ee-abed-e26f295154d8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:16.691035 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d815f54-0bb9-49ee-abed-e26f295154d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:16.693267 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d815f54-0bb9-49ee-abed-e26f295154d8 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:16.714782 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 406/822] 10.222.0.13 () {56 vars in 1135 bytes} [Fri Jun 24 08:28:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:16.715561 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d815f54-0bb9-49ee-abed-e26f295154d8 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:16.763993 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a860eb15-627a-46bc-a544-30fe8eb4f044 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:16.796755 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:16.801035 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3f034d-f4ba-4c7b-b901-9762105961fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:16.801662 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f034d-f4ba-4c7b-b901-9762105961fc None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:16.801946 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f034d-f4ba-4c7b-b901-9762105961fc None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:16.802216 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f034d-f4ba-4c7b-b901-9762105961fc None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:16.802916 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3f034d-f4ba-4c7b-b901-9762105961fc None admin] RBAC: Adding query filter params (type=network) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:16.803229 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3f034d-f4ba-4c7b-b901-9762105961fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:16.804656 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3f034d-f4ba-4c7b-b901-9762105961fc None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:16.814447 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 417/823] 10.222.0.13 () {56 vars in 1112 bytes} [Fri Jun 24 08:28:16 2022] GET /identity/v3/services?type=network => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:16.814447 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3f034d-f4ba-4c7b-b901-9762105961fc None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:16.823512 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d815f54-0bb9-49ee-abed-e26f295154d8 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:16.827123 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:16.860160 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a654b424-d678-48ff-9bd4-b1d6a54ccef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:16.860739 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a654b424-d678-48ff-9bd4-b1d6a54ccef5 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:16.861020 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a654b424-d678-48ff-9bd4-b1d6a54ccef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:16.861301 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a654b424-d678-48ff-9bd4-b1d6a54ccef5 None admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:16.862113 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a654b424-d678-48ff-9bd4-b1d6a54ccef5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:16.862424 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a654b424-d678-48ff-9bd4-b1d6a54ccef5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:16.863858 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a654b424-d678-48ff-9bd4-b1d6a54ccef5 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:16.877272 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 407/824] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Jun 24 08:28:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:16.878209 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a654b424-d678-48ff-9bd4-b1d6a54ccef5 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:16.933493 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3f034d-f4ba-4c7b-b901-9762105961fc None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:16.971109 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139cb5a6-84ba-4272-b062-e6de4b4c4a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:16.972133 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139cb5a6-84ba-4272-b062-e6de4b4c4a66 None admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:16.972415 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139cb5a6-84ba-4272-b062-e6de4b4c4a66 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:16.972688 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139cb5a6-84ba-4272-b062-e6de4b4c4a66 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:16.973419 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139cb5a6-84ba-4272-b062-e6de4b4c4a66 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:16.974808 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139cb5a6-84ba-4272-b062-e6de4b4c4a66 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:16.985811 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a654b424-d678-48ff-9bd4-b1d6a54ccef5 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:17.005717 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 418/825] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Jun 24 08:28:16 2022] POST /identity/v3/endpoints => generated 346 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:28:17.005717 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139cb5a6-84ba-4272-b062-e6de4b4c4a66 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:17.030073 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:17.035926 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc15dc6-e15c-457e-9148-86b81602e5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:17.037010 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc15dc6-e15c-457e-9148-86b81602e5d9 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:17.037299 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc15dc6-e15c-457e-9148-86b81602e5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:17.037581 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc15dc6-e15c-457e-9148-86b81602e5d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:17.056340 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc15dc6-e15c-457e-9148-86b81602e5d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:17.057864 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc15dc6-e15c-457e-9148-86b81602e5d9 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:17.076626 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-2cc15dc6-e15c-457e-9148-86b81602e5d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:17.076626 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:17.076626 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:17.076626 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:17.076626 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:17.076626 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:17.076626 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:17.076626 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:17.076626 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:17.076626 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:17.076626 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:17.076626 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:17.076626 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:28:17.076626 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:28:17.076626 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:28:17.077786 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:28:17.077786 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:17.077786 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:17.077786 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:17.077786 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:17.077786 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:17.077786 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:17.077786 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:17.077786 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:17.077786 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:17.077786 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:17.077786 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:17.077786 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:17.077786 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:17.077786 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:17.077786 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:17.077786 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:17.079026 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:28:17.079026 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:28:17.079026 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:17.079026 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:17.079026 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 408/826] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:17.079368 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc15dc6-e15c-457e-9148-86b81602e5d9 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:17.106001 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:17.163332 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139cb5a6-84ba-4272-b062-e6de4b4c4a66 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:17.206750 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13da4c1e-b873-4ee8-82a4-619562500301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:17.207509 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da4c1e-b873-4ee8-82a4-619562500301 None admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:17.207864 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da4c1e-b873-4ee8-82a4-619562500301 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:17.208209 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da4c1e-b873-4ee8-82a4-619562500301 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:17.208589 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc15dc6-e15c-457e-9148-86b81602e5d9 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:17.209033 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13da4c1e-b873-4ee8-82a4-619562500301 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:17.210514 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13da4c1e-b873-4ee8-82a4-619562500301 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:17.241781 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 419/827] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Jun 24 08:28:16 2022] POST /identity/v3/endpoints => generated 327 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:28:17.242756 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13da4c1e-b873-4ee8-82a4-619562500301 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:17.272076 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712760ee-e711-4eb2-9f9c-bdfaeadbdc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:17.272757 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712760ee-e711-4eb2-9f9c-bdfaeadbdc10 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:17.273037 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712760ee-e711-4eb2-9f9c-bdfaeadbdc10 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:17.273315 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712760ee-e711-4eb2-9f9c-bdfaeadbdc10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:17.274242 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712760ee-e711-4eb2-9f9c-bdfaeadbdc10 None admin] RBAC: Adding query filter params (role.id=acfe707eee6944ad94d1915fc9ee7ea3, scope.project.id=180b0af5863349e99ae412b07c1e9465, user.id=74bb1bc6821b4500b26f160ccae5b7d5) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:17.274560 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712760ee-e711-4eb2-9f9c-bdfaeadbdc10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:17.276492 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712760ee-e711-4eb2-9f9c-bdfaeadbdc10 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:17.297150 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 409/828] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Jun 24 08:28:17 2022] GET /identity/v3/role_assignments?user.id=74bb1bc6821b4500b26f160ccae5b7d5&scope.project.id=180b0af5863349e99ae412b07c1e9465&role.id=acfe707eee6944ad94d1915fc9ee7ea3 => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:17.392981 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13da4c1e-b873-4ee8-82a4-619562500301 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:17.420656 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:17.436513 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb3e40e-e7a7-4d49-9393-4d6498858b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:17.437203 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3e40e-e7a7-4d49-9393-4d6498858b6f None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:17.437538 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3e40e-e7a7-4d49-9393-4d6498858b6f None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:17.442318 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3e40e-e7a7-4d49-9393-4d6498858b6f None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:17.443175 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb3e40e-e7a7-4d49-9393-4d6498858b6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:17.446718 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb3e40e-e7a7-4d49-9393-4d6498858b6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:17.446718 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb3e40e-e7a7-4d49-9393-4d6498858b6f None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:17.465318 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 420/829] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:28:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:17.470633 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712760ee-e711-4eb2-9f9c-bdfaeadbdc10 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:17.523269 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:17.592322 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712760ee-e711-4eb2-9f9c-bdfaeadbdc10 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:17.631963 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c58c3a-6533-49c7-92c2-f6b20b249c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:17.632755 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c58c3a-6533-49c7-92c2-f6b20b249c4c None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:17.633038 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c58c3a-6533-49c7-92c2-f6b20b249c4c None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:17.633379 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c58c3a-6533-49c7-92c2-f6b20b249c4c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:17.643446 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c58c3a-6533-49c7-92c2-f6b20b249c4c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:17.644906 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c58c3a-6533-49c7-92c2-f6b20b249c4c None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:17.661918 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-90c58c3a-6533-49c7-92c2-f6b20b249c4c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:28:17.661918 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:17.661918 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:17.661918 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:17.661918 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:17.661918 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:17.661918 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:17.661918 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:17.661918 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:17.661918 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:17.661918 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:17.661918 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:17.661918 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:28:17.661918 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:28:17.661918 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:28:17.662967 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:28:17.662967 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:17.662967 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:17.662967 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:17.662967 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:17.662967 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:17.662967 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:17.662967 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:17.662967 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:17.662967 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:17.662967 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:17.662967 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:17.662967 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:17.662967 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:17.662967 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:17.662967 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:17.662967 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:17.663853 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:28:17.663853 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:28:17.663853 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:28:17.663853 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:28:17.663853 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:28:17.663853 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:28:17.663853 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:28:17.663853 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:17.667872 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 410/830] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:28:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:17.673437 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb3e40e-e7a7-4d49-9393-4d6498858b6f None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:17.792579 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb3e40e-e7a7-4d49-9393-4d6498858b6f None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:17.830343 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ae8c1a-edce-429b-96fe-8e36b55b9c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:17.831366 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae8c1a-edce-429b-96fe-8e36b55b9c77 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:17.831949 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae8c1a-edce-429b-96fe-8e36b55b9c77 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:17.832465 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae8c1a-edce-429b-96fe-8e36b55b9c77 None admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:17.833446 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ae8c1a-edce-429b-96fe-8e36b55b9c77 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:17.833979 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ae8c1a-edce-429b-96fe-8e36b55b9c77 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:17.835607 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ae8c1a-edce-429b-96fe-8e36b55b9c77 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:17.849519 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 421/831] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Jun 24 08:28:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:17.855831 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c58c3a-6533-49c7-92c2-f6b20b249c4c None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:17.960435 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c58c3a-6533-49c7-92c2-f6b20b249c4c None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:17.962733 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:17.997395 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d2c710-5036-46e6-81dd-f9230038e66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:17.998577 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d2c710-5036-46e6-81dd-f9230038e66d None admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:17.999066 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d2c710-5036-46e6-81dd-f9230038e66d None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:17.999510 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d2c710-5036-46e6-81dd-f9230038e66d None admin] PATH_INFO: `/v3/projects/180b0af5863349e99ae412b07c1e9465/users/bec8c2f113cf41a0939b81f38fbee813/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:18.001630 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d2c710-5036-46e6-81dd-f9230038e66d None admin] RBAC: Authorizing `identity:create_grant(project_id=180b0af5863349e99ae412b07c1e9465, user_id=bec8c2f113cf41a0939b81f38fbee813, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:18.003499 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d2c710-5036-46e6-81dd-f9230038e66d None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:18.019207 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 411/832] 10.222.0.13 () {58 vars in 1439 bytes} [Fri Jun 24 08:28:17 2022] PUT /identity/v3/projects/180b0af5863349e99ae412b07c1e9465/users/bec8c2f113cf41a0939b81f38fbee813/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:28:18.614265 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93fa2e-a5ea-487a-8124-6214a656765f None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:18.615284 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93fa2e-a5ea-487a-8124-6214a656765f None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:18.615797 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93fa2e-a5ea-487a-8124-6214a656765f None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:18.617684 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 422/833] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:18.622870 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff479b0-3f55-4fcc-bed4-113e28a8d274 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:18.623269 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff479b0-3f55-4fcc-bed4-113e28a8d274 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:18.623344 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff479b0-3f55-4fcc-bed4-113e28a8d274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:18.640713 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-2ff479b0-3f55-4fcc-bed4-113e28a8d274 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:18.641952 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-2ff479b0-3f55-4fcc-bed4-113e28a8d274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:18.713195 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 412/834] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:18 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 08:28:18.736454 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b93fa2e-a5ea-487a-8124-6214a656765f None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:18.783216 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:18.844671 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b93fa2e-a5ea-487a-8124-6214a656765f None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:18.883854 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022bb9b4-5864-403e-8c36-92f1bc5764ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:18.884686 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022bb9b4-5864-403e-8c36-92f1bc5764ac None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:18.884969 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022bb9b4-5864-403e-8c36-92f1bc5764ac None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:18.885246 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022bb9b4-5864-403e-8c36-92f1bc5764ac None admin] PATH_INFO: `/v3/roles/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:18.896096 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022bb9b4-5864-403e-8c36-92f1bc5764ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:18.904235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-022bb9b4-5864-403e-8c36-92f1bc5764ac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:28:18.904235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:18.904235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:18.904235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:18.904235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:18.904235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:18.904235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:18.904235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:18.904235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:18.904235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:18.904235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:18.904235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:18.904235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:18.904235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:18.904235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:18.904235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:18.905354 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:18.905354 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:18.905354 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:18.905354 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:18.905354 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:18.905354 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:18.905354 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:18.905354 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:18.905354 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:18.905354 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:18.905354 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:18.905354 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:18.905354 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:18.905354 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:18.905354 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:18.905354 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:18.906235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:18.906235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:18.906235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:18.906235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:18.906235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:18.906235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:18.906235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:18.906235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:18.906235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:18.906235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:18.906235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:28:18.906235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:18.906235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 423/835] 10.222.0.13 () {56 vars in 1102 bytes} [Fri Jun 24 08:28:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:18.911549 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff479b0-3f55-4fcc-bed4-113e28a8d274 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:19.011364 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:19.020196 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff479b0-3f55-4fcc-bed4-113e28a8d274 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:19.056580 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eff3b9d-ae32-4a6f-863b-7095d692f9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:19.057295 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff3b9d-ae32-4a6f-863b-7095d692f9d4 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:19.057583 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff3b9d-ae32-4a6f-863b-7095d692f9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:19.057889 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff3b9d-ae32-4a6f-863b-7095d692f9d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:19.058633 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eff3b9d-ae32-4a6f-863b-7095d692f9d4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:19.058940 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eff3b9d-ae32-4a6f-863b-7095d692f9d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:19.060315 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eff3b9d-ae32-4a6f-863b-7095d692f9d4 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:19.074099 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 413/836] 10.222.0.13 () {56 vars in 1103 bytes} [Fri Jun 24 08:28:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:19.076568 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022bb9b4-5864-403e-8c36-92f1bc5764ac None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:19.181587 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:19.183555 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f3241-0561-46a8-ae81-9ad5ecf82461 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:19.183861 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f3241-0561-46a8-ae81-9ad5ecf82461 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:19.184132 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f3241-0561-46a8-ae81-9ad5ecf82461 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:19.185806 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 414/837] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:19.190236 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022bb9b4-5864-403e-8c36-92f1bc5764ac None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:19.191057 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd5849-6d60-49d4-8319-3dc79e54b204 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:19.191554 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd5849-6d60-49d4-8319-3dc79e54b204 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:19.192055 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd5849-6d60-49d4-8319-3dc79e54b204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:19.212009 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-40fd5849-6d60-49d4-8319-3dc79e54b204 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:19.213875 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-40fd5849-6d60-49d4-8319-3dc79e54b204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:19.235695 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f10d1d-7894-4c9d-a50c-0bcf37325072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:19.236488 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f10d1d-7894-4c9d-a50c-0bcf37325072 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:19.236772 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f10d1d-7894-4c9d-a50c-0bcf37325072 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:19.237050 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f10d1d-7894-4c9d-a50c-0bcf37325072 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:19.251444 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f10d1d-7894-4c9d-a50c-0bcf37325072 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:19.253159 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f10d1d-7894-4c9d-a50c-0bcf37325072 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:19.279175 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-c6f10d1d-7894-4c9d-a50c-0bcf37325072 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:19.279175 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:19.279175 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:19.279175 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:19.279175 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:19.279175 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:19.279175 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:19.279175 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:19.279175 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:19.279175 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:19.279175 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:19.279175 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:19.279175 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:28:19.279175 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:28:19.279175 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:28:19.280137 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:28:19.280137 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:19.280137 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:19.280137 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:19.280137 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:19.280137 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:19.280137 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:19.280137 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:19.280137 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:19.280137 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:19.280137 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:19.280137 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:19.280137 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:19.280137 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:19.280137 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:19.280137 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:19.280137 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:19.281025 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:28:19.281025 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:28:19.281025 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:19.281025 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:19.285226 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 424/838] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:19.286980 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f10d1d-7894-4c9d-a50c-0bcf37325072 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:19.305720 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 415/839] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:19 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 08:28:19.323350 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fd5849-6d60-49d4-8319-3dc79e54b204 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:19.402814 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f10d1d-7894-4c9d-a50c-0bcf37325072 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:19.413432 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:19.432210 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fd5849-6d60-49d4-8319-3dc79e54b204 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:19.438323 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34510ef-0d0e-4305-a48f-5cc213801c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:19.438984 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34510ef-0d0e-4305-a48f-5cc213801c08 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:19.439265 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34510ef-0d0e-4305-a48f-5cc213801c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:19.439541 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34510ef-0d0e-4305-a48f-5cc213801c08 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:19.440255 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34510ef-0d0e-4305-a48f-5cc213801c08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:19.440563 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34510ef-0d0e-4305-a48f-5cc213801c08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:19.442052 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34510ef-0d0e-4305-a48f-5cc213801c08 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:19.457012 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 425/840] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:28:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:19.465279 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34510ef-0d0e-4305-a48f-5cc213801c08 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:19.475945 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd86ecb7-8112-448e-9ba2-cc3dca7bea45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:19.478556 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd86ecb7-8112-448e-9ba2-cc3dca7bea45 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:19.478859 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd86ecb7-8112-448e-9ba2-cc3dca7bea45 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:19.479157 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd86ecb7-8112-448e-9ba2-cc3dca7bea45 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:19.479979 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd86ecb7-8112-448e-9ba2-cc3dca7bea45 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:19.481885 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd86ecb7-8112-448e-9ba2-cc3dca7bea45 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:19.485778 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:19.500058 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-dd86ecb7-8112-448e-9ba2-cc3dca7bea45 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 08:28:19.500058 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:19.500058 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 08:28:19.500058 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 08:28:19.500058 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 08:28:19.500058 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 08:28:19.500058 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 08:28:19.500058 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:28:19.500058 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 08:28:19.500058 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:19.500058 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:28:19.500058 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:19.500058 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:19.500058 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:19.500058 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:19.500058 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:19.500058 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:19.501050 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:19.501050 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:19.501050 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:19.501050 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:19.501050 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:19.501050 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:19.501050 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 08:28:19.501050 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 08:28:19.501050 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 08:28:19.501050 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 08:28:19.501050 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:19.501050 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:19.501050 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:19.501050 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:19.501050 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:19.501050 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:19.502502 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:19.502502 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:19.502502 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:19.502502 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:19.502502 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:19.502502 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:19.502502 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:19.502502 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:19.502502 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:19.502502 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:19.502502 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 08:28:19.502502 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:28:19.502502 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 08:28:19.502502 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:19.502502 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 416/841] 10.222.0.13 () {56 vars in 1114 bytes} [Fri Jun 24 08:28:19 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:19.507184 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd86ecb7-8112-448e-9ba2-cc3dca7bea45 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:19.590657 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34510ef-0d0e-4305-a48f-5cc213801c08 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:19.628392 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f2f93b-f14a-4674-94c9-233091c07362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:19.630023 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2f93b-f14a-4674-94c9-233091c07362 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:19.630511 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2f93b-f14a-4674-94c9-233091c07362 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:19.631102 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2f93b-f14a-4674-94c9-233091c07362 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:19.644454 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f2f93b-f14a-4674-94c9-233091c07362 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:19.645593 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd86ecb7-8112-448e-9ba2-cc3dca7bea45 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:19.646837 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f2f93b-f14a-4674-94c9-233091c07362 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:19.667580 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-50f2f93b-f14a-4674-94c9-233091c07362 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 08:28:19.667580 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:19.667580 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:19.667580 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:19.667580 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:19.667580 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:19.667580 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:19.667580 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:19.667580 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:19.667580 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:19.667580 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:19.667580 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:19.667580 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:28:19.667580 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:28:19.667580 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:28:19.667580 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:28:19.668569 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:19.668569 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:19.668569 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:28:19.668569 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:28:19.668569 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:28:19.668569 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:28:19.668569 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:19.668569 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:19.668569 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:19.668569 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:19.668569 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:19.668569 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:19.668569 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:19.668569 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:19.668569 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:19.668569 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:19.668569 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:19.668569 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:19.669538 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:19.669538 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:19.669538 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:28:19.669538 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:28:19.669538 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:28:19.669538 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:28:19.669538 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:28:19.669538 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:28:19.669538 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 08:28:19.669538 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:19.678024 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 426/842] 10.222.0.13 () {56 vars in 1117 bytes} [Fri Jun 24 08:28:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:19.679332 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f2f93b-f14a-4674-94c9-233091c07362 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:19.691521 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1934cb2-9113-41de-b924-f378f6b103b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:19.692113 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1934cb2-9113-41de-b924-f378f6b103b2 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:19.692396 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1934cb2-9113-41de-b924-f378f6b103b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:19.692671 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1934cb2-9113-41de-b924-f378f6b103b2 None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:19.693379 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1934cb2-9113-41de-b924-f378f6b103b2 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:19.694948 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1934cb2-9113-41de-b924-f378f6b103b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:19.695058 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1934cb2-9113-41de-b924-f378f6b103b2 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:19.708756 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 417/843] 10.222.0.13 () {56 vars in 1114 bytes} [Fri Jun 24 08:28:19 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:19.708756 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1934cb2-9113-41de-b924-f378f6b103b2 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:19.730910 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:19.787225 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:19.820796 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f2f93b-f14a-4674-94c9-233091c07362 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:19.849281 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1934cb2-9113-41de-b924-f378f6b103b2 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:19.857516 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b82dfb2-06c8-4e14-bb41-40b18644478a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:19.858224 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b82dfb2-06c8-4e14-bb41-40b18644478a None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:19.858506 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b82dfb2-06c8-4e14-bb41-40b18644478a None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:19.858782 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b82dfb2-06c8-4e14-bb41-40b18644478a None admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:19.859616 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b82dfb2-06c8-4e14-bb41-40b18644478a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:19.859922 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b82dfb2-06c8-4e14-bb41-40b18644478a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:19.861343 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b82dfb2-06c8-4e14-bb41-40b18644478a None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:19.880111 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 427/844] 10.222.0.13 () {56 vars in 1149 bytes} [Fri Jun 24 08:28:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:19.880111 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc26fd-f157-4190-975c-af61949ecde9 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:19.880111 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc26fd-f157-4190-975c-af61949ecde9 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:19.880111 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc26fd-f157-4190-975c-af61949ecde9 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:19.880111 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 428/845] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:19.886829 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc26fd-f157-4190-975c-af61949ecde9 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:19.891562 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4365da-3855-4c8d-8428-a40e69eeded0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:19.892460 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4365da-3855-4c8d-8428-a40e69eeded0 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:19.893041 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4365da-3855-4c8d-8428-a40e69eeded0 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:19.893560 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4365da-3855-4c8d-8428-a40e69eeded0 None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:19.894526 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4365da-3855-4c8d-8428-a40e69eeded0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:19.895076 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4365da-3855-4c8d-8428-a40e69eeded0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:19.896722 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4365da-3855-4c8d-8428-a40e69eeded0 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:19.902667 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 418/846] 10.222.0.13 () {56 vars in 1114 bytes} [Fri Jun 24 08:28:19 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:19.904048 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfbafba-b29d-438b-99e0-5f9e0d4f38ed None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:19.904512 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfbafba-b29d-438b-99e0-5f9e0d4f38ed None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:19.904963 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfbafba-b29d-438b-99e0-5f9e0d4f38ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:19.923414 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-7bfbafba-b29d-438b-99e0-5f9e0d4f38ed None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:19.924615 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-7bfbafba-b29d-438b-99e0-5f9e0d4f38ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:20.020633 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 419/847] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:19 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:28:20.042803 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfbafba-b29d-438b-99e0-5f9e0d4f38ed None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:20.043704 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc26fd-f157-4190-975c-af61949ecde9 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:20.054563 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:20.080894 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:20.087886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478fe6a7-1647-43f2-b2d7-ac267d076c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:20.088552 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478fe6a7-1647-43f2-b2d7-ac267d076c13 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:20.089118 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478fe6a7-1647-43f2-b2d7-ac267d076c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:20.089715 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478fe6a7-1647-43f2-b2d7-ac267d076c13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:20.172196 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478fe6a7-1647-43f2-b2d7-ac267d076c13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:20.173707 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478fe6a7-1647-43f2-b2d7-ac267d076c13 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:20.173978 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfbafba-b29d-438b-99e0-5f9e0d4f38ed None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:20.213558 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d221fdf-689c-45c7-845b-4865d43cb031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:20.214398 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d221fdf-689c-45c7-845b-4865d43cb031 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:20.214822 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d221fdf-689c-45c7-845b-4865d43cb031 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:20.215004 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d221fdf-689c-45c7-845b-4865d43cb031 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:20.215581 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d221fdf-689c-45c7-845b-4865d43cb031 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:20.216963 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d221fdf-689c-45c7-845b-4865d43cb031 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:20.230399 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-8d221fdf-689c-45c7-845b-4865d43cb031 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 08:28:20.230399 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:20.230399 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 08:28:20.230399 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 08:28:20.230399 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 08:28:20.230399 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 08:28:20.230399 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 08:28:20.230399 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:28:20.230399 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 08:28:20.230399 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:20.230399 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:28:20.230399 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:20.230399 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:20.230399 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:20.230399 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:20.230399 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:20.231360 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:20.231360 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:20.231360 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:20.231360 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:20.231360 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:20.231360 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:20.231360 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:20.231360 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 08:28:20.231360 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 08:28:20.231360 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 08:28:20.231360 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 08:28:20.231360 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:20.231360 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:20.231360 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:20.231360 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:20.231360 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:20.232201 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:20.232201 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:20.232201 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:20.232201 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:20.232201 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:20.232201 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:20.232201 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:20.232201 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:20.232201 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:20.232201 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:20.232201 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:20.232201 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 08:28:20.232201 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:28:20.232201 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 08:28:20.232201 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:20.232201 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 420/848] 10.222.0.13 () {56 vars in 1132 bytes} [Fri Jun 24 08:28:20 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:28:20.233072 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b976613f-f204-473f-ad05-e3cd6e29f654 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:20.233072 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b976613f-f204-473f-ad05-e3cd6e29f654 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:20.233797 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b976613f-f204-473f-ad05-e3cd6e29f654 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:20.234852 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 421/849] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:20.237159 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b976613f-f204-473f-ad05-e3cd6e29f654 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:20.294128 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 429/850] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:28:19 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 08:28:20.295637 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16cd1ce-acd9-4764-b24d-8148be3e54bc None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:20.295922 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16cd1ce-acd9-4764-b24d-8148be3e54bc None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:20.296192 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16cd1ce-acd9-4764-b24d-8148be3e54bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:20.313591 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-f16cd1ce-acd9-4764-b24d-8148be3e54bc None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:20.314833 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-f16cd1ce-acd9-4764-b24d-8148be3e54bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:20.324946 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:20.340908 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:20.368371 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b976613f-f204-473f-ad05-e3cd6e29f654 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:20.391992 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 430/851] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:20 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:28:20.392834 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16cd1ce-acd9-4764-b24d-8148be3e54bc None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:20.413381 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e68f8c-b389-4f86-8764-5d8d2fafb059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:20.415030 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e68f8c-b389-4f86-8764-5d8d2fafb059 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:20.415554 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e68f8c-b389-4f86-8764-5d8d2fafb059 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:20.416119 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e68f8c-b389-4f86-8764-5d8d2fafb059 None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:20.417014 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e68f8c-b389-4f86-8764-5d8d2fafb059 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:20.417506 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e68f8c-b389-4f86-8764-5d8d2fafb059 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:20.419475 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e68f8c-b389-4f86-8764-5d8d2fafb059 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:20.424460 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 422/852] 10.222.0.13 () {56 vars in 1126 bytes} [Fri Jun 24 08:28:20 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:20.425241 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e68f8c-b389-4f86-8764-5d8d2fafb059 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:20.511524 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16cd1ce-acd9-4764-b24d-8148be3e54bc None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:20.533393 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e68f8c-b389-4f86-8764-5d8d2fafb059 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:20.549061 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0a1351-ce7e-4a2b-9c4c-c823ec41bbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:20.549919 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a1351-ce7e-4a2b-9c4c-c823ec41bbb3 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:20.550207 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a1351-ce7e-4a2b-9c4c-c823ec41bbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:20.550483 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a1351-ce7e-4a2b-9c4c-c823ec41bbb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:20.561590 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0a1351-ce7e-4a2b-9c4c-c823ec41bbb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:20.563166 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0a1351-ce7e-4a2b-9c4c-c823ec41bbb3 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:20.570089 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b2276c-cfd6-439a-9b86-ebfd3edf2a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:20.571878 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b2276c-cfd6-439a-9b86-ebfd3edf2a77 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:20.572550 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b2276c-cfd6-439a-9b86-ebfd3edf2a77 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:20.572832 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b2276c-cfd6-439a-9b86-ebfd3edf2a77 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:20.574823 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:20.581421 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-1d0a1351-ce7e-4a2b-9c4c-c823ec41bbb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:20.581421 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:20.581421 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:20.581421 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:20.581421 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:20.581421 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:20.581421 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:20.581421 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:20.581421 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:20.581421 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:20.581421 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:20.581421 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:20.581421 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:28:20.581421 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:28:20.581421 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:28:20.582472 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:28:20.582472 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:20.582472 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:20.582472 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:20.582472 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:20.582472 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:20.582472 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:20.582472 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:20.582472 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:20.582472 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:20.582472 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:20.582472 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:20.582472 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:20.582472 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:20.582472 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:20.582472 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:20.582472 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:20.583407 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:28:20.583407 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:28:20.583407 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:20.583407 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:20.583407 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 431/853] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:20.583674 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0a1351-ce7e-4a2b-9c4c-c823ec41bbb3 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:20.592075 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b2276c-cfd6-439a-9b86-ebfd3edf2a77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:20.598831 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:20.599642 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-91b2276c-cfd6-439a-9b86-ebfd3edf2a77 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:28:20.599642 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:20.599642 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:20.599642 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:20.599642 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:20.599642 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:20.599642 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:20.599642 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:20.599642 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:20.599642 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:20.599642 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:20.599642 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:20.599642 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:20.599642 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:20.599642 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:20.599642 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:20.600560 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:20.600560 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:20.600560 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:20.600560 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:20.600560 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:20.600560 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:20.600560 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:20.600560 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:20.600560 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:20.600560 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:20.600560 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:20.600560 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:20.600560 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:20.600560 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:20.600560 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:20.600560 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:20.607353 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:20.607353 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:20.607353 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:20.607353 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:20.607353 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:20.607353 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:20.607353 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:20.607353 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:20.607353 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:20.607353 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:20.607353 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:28:20.607353 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:20.607353 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 423/854] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Jun 24 08:28:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:20.607353 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b2276c-cfd6-439a-9b86-ebfd3edf2a77 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:20.698530 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0a1351-ce7e-4a2b-9c4c-c823ec41bbb3 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:20.719095 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b2276c-cfd6-439a-9b86-ebfd3edf2a77 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:20.734017 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cc0c9a-d060-4e43-8f09-0f5f00446ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:20.734679 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc0c9a-d060-4e43-8f09-0f5f00446ada None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:20.734963 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc0c9a-d060-4e43-8f09-0f5f00446ada None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:20.735238 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc0c9a-d060-4e43-8f09-0f5f00446ada None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:20.735944 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cc0c9a-d060-4e43-8f09-0f5f00446ada None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:20.736249 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cc0c9a-d060-4e43-8f09-0f5f00446ada None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:20.737602 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cc0c9a-d060-4e43-8f09-0f5f00446ada None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:20.742372 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 432/855] 10.222.0.13 () {56 vars in 1126 bytes} [Fri Jun 24 08:28:20 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:20.743533 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cc0c9a-d060-4e43-8f09-0f5f00446ada None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:20.775954 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daedf2d-2eb2-4801-b443-9fd45ce48135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:20.776625 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daedf2d-2eb2-4801-b443-9fd45ce48135 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:20.776908 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daedf2d-2eb2-4801-b443-9fd45ce48135 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:20.777183 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daedf2d-2eb2-4801-b443-9fd45ce48135 None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:20.777937 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daedf2d-2eb2-4801-b443-9fd45ce48135 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:20.778242 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daedf2d-2eb2-4801-b443-9fd45ce48135 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:20.779655 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daedf2d-2eb2-4801-b443-9fd45ce48135 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:20.801061 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 424/856] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:28:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:20.801990 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daedf2d-2eb2-4801-b443-9fd45ce48135 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:20.864421 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:20.870063 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:20.900818 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cc0c9a-d060-4e43-8f09-0f5f00446ada None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:20.924386 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daedf2d-2eb2-4801-b443-9fd45ce48135 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:20.942184 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82823f99-e6d5-4b6d-a09b-9cfc0fa48d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:20.942860 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82823f99-e6d5-4b6d-a09b-9cfc0fa48d8f None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:20.943146 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82823f99-e6d5-4b6d-a09b-9cfc0fa48d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:20.943430 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82823f99-e6d5-4b6d-a09b-9cfc0fa48d8f None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:20.944183 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82823f99-e6d5-4b6d-a09b-9cfc0fa48d8f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:20.944493 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82823f99-e6d5-4b6d-a09b-9cfc0fa48d8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:20.946544 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82823f99-e6d5-4b6d-a09b-9cfc0fa48d8f None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:20.960588 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 433/857] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:28:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:20.962850 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82823f99-e6d5-4b6d-a09b-9cfc0fa48d8f None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:20.979378 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e48be3-cb50-477c-929a-4b90a4c801f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:20.980662 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e48be3-cb50-477c-929a-4b90a4c801f5 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:20.981546 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e48be3-cb50-477c-929a-4b90a4c801f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:20.982015 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e48be3-cb50-477c-929a-4b90a4c801f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:20.991595 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e48be3-cb50-477c-929a-4b90a4c801f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:20.993492 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e48be3-cb50-477c-929a-4b90a4c801f5 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:21.013551 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-b6e48be3-cb50-477c-929a-4b90a4c801f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:28:21.013551 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:21.013551 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:21.013551 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:21.013551 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:21.013551 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:21.013551 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:21.013551 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:21.013551 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:21.013551 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:21.013551 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:21.013551 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:21.013551 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:28:21.013551 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:28:21.013551 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:28:21.014596 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:28:21.014596 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:21.014596 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:21.014596 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:21.014596 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:21.014596 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:21.014596 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:21.014596 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:21.014596 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:21.014596 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:21.014596 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:21.014596 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:21.014596 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:21.014596 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:21.014596 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:21.014596 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:21.014596 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:21.015641 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:28:21.015641 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:28:21.015641 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:28:21.015641 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:28:21.015641 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:28:21.015641 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:28:21.015641 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:28:21.015641 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:21.018088 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 425/858] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:28:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:21.023507 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e48be3-cb50-477c-929a-4b90a4c801f5 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:21.081537 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82823f99-e6d5-4b6d-a09b-9cfc0fa48d8f None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:21.124243 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb70e1-3b80-4d13-9dd1-a49242c79ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:21.125330 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb70e1-3b80-4d13-9dd1-a49242c79ba0 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:21.125615 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb70e1-3b80-4d13-9dd1-a49242c79ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:21.125927 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb70e1-3b80-4d13-9dd1-a49242c79ba0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:21.135558 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:21.142729 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bb70e1-3b80-4d13-9dd1-a49242c79ba0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:21.144749 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bb70e1-3b80-4d13-9dd1-a49242c79ba0 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:21.167528 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e48be3-cb50-477c-929a-4b90a4c801f5 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:21.168354 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-c2bb70e1-3b80-4d13-9dd1-a49242c79ba0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:21.168354 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:21.168354 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:21.168354 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:21.168354 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:21.168354 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:21.168354 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:21.168354 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:21.168354 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:21.168354 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:21.168354 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:21.168354 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:21.168354 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:28:21.168354 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:28:21.168354 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:28:21.169223 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:28:21.169223 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:21.169223 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:21.169223 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:21.169223 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:21.169223 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:21.169223 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:21.169223 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:21.169223 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:21.169223 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:21.169223 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:21.169223 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:21.169223 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:21.169223 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:21.169223 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:21.169223 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:21.169223 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:21.170568 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:28:21.170568 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:28:21.170568 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:21.170568 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:21.170568 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 434/859] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:21.175057 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb70e1-3b80-4d13-9dd1-a49242c79ba0 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:21.191031 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:21.219990 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40421d3d-3676-4355-b3aa-ce2da50ae9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:21.220930 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40421d3d-3676-4355-b3aa-ce2da50ae9d9 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:21.221363 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40421d3d-3676-4355-b3aa-ce2da50ae9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:21.222232 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40421d3d-3676-4355-b3aa-ce2da50ae9d9 None admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:21.223173 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40421d3d-3676-4355-b3aa-ce2da50ae9d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:21.223650 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40421d3d-3676-4355-b3aa-ce2da50ae9d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:21.225942 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40421d3d-3676-4355-b3aa-ce2da50ae9d9 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:21.246074 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 426/860] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Jun 24 08:28:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:21.251814 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40421d3d-3676-4355-b3aa-ce2da50ae9d9 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:21.304907 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb70e1-3b80-4d13-9dd1-a49242c79ba0 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:21.355971 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ab923f-fbcb-4368-b6bf-e89b591aba9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:21.357006 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ab923f-fbcb-4368-b6bf-e89b591aba9b None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:21.357487 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ab923f-fbcb-4368-b6bf-e89b591aba9b None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:21.357984 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ab923f-fbcb-4368-b6bf-e89b591aba9b None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:21.358909 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ab923f-fbcb-4368-b6bf-e89b591aba9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:21.359435 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ab923f-fbcb-4368-b6bf-e89b591aba9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:21.361719 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ab923f-fbcb-4368-b6bf-e89b591aba9b None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:21.376343 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 435/861] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:28:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:21.384211 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ab923f-fbcb-4368-b6bf-e89b591aba9b None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:21.402520 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40421d3d-3676-4355-b3aa-ce2da50ae9d9 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:21.448568 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a81bf32-6eff-44a0-8904-19d07485d4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:21.449779 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a81bf32-6eff-44a0-8904-19d07485d4aa None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:21.450497 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:21.451353 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a81bf32-6eff-44a0-8904-19d07485d4aa None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:21.451804 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a81bf32-6eff-44a0-8904-19d07485d4aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:21.452831 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a81bf32-6eff-44a0-8904-19d07485d4aa None admin] RBAC: Adding query filter params (role.id=6ca1ae4292d9414ca6526c6e8608de9e, scope.project.id=180b0af5863349e99ae412b07c1e9465, user.id=85d9c4c57681447c898ebfd7990b05b6) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:21.453315 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a81bf32-6eff-44a0-8904-19d07485d4aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:21.455707 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a81bf32-6eff-44a0-8904-19d07485d4aa None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:21.470002 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 427/862] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Jun 24 08:28:21 2022] GET /identity/v3/role_assignments?user.id=85d9c4c57681447c898ebfd7990b05b6&scope.project.id=180b0af5863349e99ae412b07c1e9465&role.id=6ca1ae4292d9414ca6526c6e8608de9e => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:21.520183 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ab923f-fbcb-4368-b6bf-e89b591aba9b None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:21.562135 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493c8f24-0fc7-40c3-8c33-a245a3db73b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:21.562955 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c8f24-0fc7-40c3-8c33-a245a3db73b3 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:21.563239 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c8f24-0fc7-40c3-8c33-a245a3db73b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:21.563515 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c8f24-0fc7-40c3-8c33-a245a3db73b3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:21.573318 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493c8f24-0fc7-40c3-8c33-a245a3db73b3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:21.575002 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493c8f24-0fc7-40c3-8c33-a245a3db73b3 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:21.592298 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-493c8f24-0fc7-40c3-8c33-a245a3db73b3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 08:28:21.592298 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:21.592298 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:21.592298 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:21.592298 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:21.592298 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:21.592298 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:21.592298 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:21.592298 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:21.592298 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:21.592298 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:21.592298 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:21.592298 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:28:21.592298 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:28:21.592298 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:28:21.592298 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:28:21.593301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:21.593301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:21.593301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:28:21.593301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:28:21.593301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:28:21.593301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:28:21.593301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:21.593301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:21.593301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:21.593301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:21.593301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:21.593301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:21.593301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:21.593301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:21.593301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:21.593301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:21.593301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:21.593301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:21.594257 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:21.594257 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:21.594257 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:28:21.594257 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:28:21.594257 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:28:21.594257 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:28:21.594257 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:28:21.594257 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:28:21.594257 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 08:28:21.594257 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:21.594257 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 436/863] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Jun 24 08:28:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:28:21.599689 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a81bf32-6eff-44a0-8904-19d07485d4aa None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:21.647020 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:21.721199 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a81bf32-6eff-44a0-8904-19d07485d4aa None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:21.760163 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d02c9e7-4b13-461c-9437-2640ab3390ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:21.760842 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d02c9e7-4b13-461c-9437-2640ab3390ab None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:21.761128 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d02c9e7-4b13-461c-9437-2640ab3390ab None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:21.761402 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d02c9e7-4b13-461c-9437-2640ab3390ab None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:21.762277 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d02c9e7-4b13-461c-9437-2640ab3390ab None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:21.762648 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d02c9e7-4b13-461c-9437-2640ab3390ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:21.764146 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d02c9e7-4b13-461c-9437-2640ab3390ab None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:21.788666 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 428/864] 10.222.0.13 () {56 vars in 1135 bytes} [Fri Jun 24 08:28:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:21.795968 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493c8f24-0fc7-40c3-8c33-a245a3db73b3 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:21.905226 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493c8f24-0fc7-40c3-8c33-a245a3db73b3 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:21.917179 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:21.942112 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e4fa7-4e12-4252-b647-d26991a08737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:21.942762 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e4fa7-4e12-4252-b647-d26991a08737 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:21.943044 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e4fa7-4e12-4252-b647-d26991a08737 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:21.943317 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e4fa7-4e12-4252-b647-d26991a08737 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:22.027054 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1e4fa7-4e12-4252-b647-d26991a08737 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:22.028859 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1e4fa7-4e12-4252-b647-d26991a08737 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:22.062158 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ce9d0-7a3d-4aaf-8fb4-3fc60475c6e7 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:22.063101 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ce9d0-7a3d-4aaf-8fb4-3fc60475c6e7 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:22.063850 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ce9d0-7a3d-4aaf-8fb4-3fc60475c6e7 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:22.068184 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 429/865] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:28:22.075195 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db3cf3-f1de-4fc4-9962-1ab2033c2649 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:22.075481 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db3cf3-f1de-4fc4-9962-1ab2033c2649 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:22.075763 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db3cf3-f1de-4fc4-9962-1ab2033c2649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:22.093926 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-89db3cf3-f1de-4fc4-9962-1ab2033c2649 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:22.095236 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-89db3cf3-f1de-4fc4-9962-1ab2033c2649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:22.149653 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 437/866] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:28:21 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 24 08:28:22.150988 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e4fa7-4e12-4252-b647-d26991a08737 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:22.162355 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:22.182135 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 430/867] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:22 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 08:28:22.199140 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89db3cf3-f1de-4fc4-9962-1ab2033c2649 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:22.232728 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:22.255306 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e4fa7-4e12-4252-b647-d26991a08737 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:22.297118 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400a3f47-f161-4a05-afa6-73ffc91acd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:22.298343 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400a3f47-f161-4a05-afa6-73ffc91acd45 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:22.299325 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400a3f47-f161-4a05-afa6-73ffc91acd45 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:22.299798 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400a3f47-f161-4a05-afa6-73ffc91acd45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:22.318931 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400a3f47-f161-4a05-afa6-73ffc91acd45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:22.320636 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400a3f47-f161-4a05-afa6-73ffc91acd45 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:22.332672 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89db3cf3-f1de-4fc4-9962-1ab2033c2649 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:22.340855 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-400a3f47-f161-4a05-afa6-73ffc91acd45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:22.340855 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:22.340855 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:22.340855 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:22.340855 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:22.340855 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:22.340855 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:22.340855 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:22.340855 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:22.340855 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:22.340855 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:22.340855 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:22.340855 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:28:22.340855 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:28:22.340855 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:28:22.341834 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:28:22.341834 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:22.341834 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:22.341834 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:22.341834 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:22.341834 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:22.341834 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:22.341834 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:22.341834 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:22.341834 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:22.341834 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:22.341834 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:22.341834 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:22.341834 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:22.341834 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:22.341834 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:22.341834 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:22.342727 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:28:22.342727 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:28:22.342727 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:22.342727 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:22.346809 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 438/868] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:28:22.348069 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400a3f47-f161-4a05-afa6-73ffc91acd45 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:22.392502 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09bdb9e-4e75-4bb6-9b1f-0afc26f251b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:22.393804 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09bdb9e-4e75-4bb6-9b1f-0afc26f251b8 None admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:22.394551 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09bdb9e-4e75-4bb6-9b1f-0afc26f251b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:22.395026 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09bdb9e-4e75-4bb6-9b1f-0afc26f251b8 None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:22.396588 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09bdb9e-4e75-4bb6-9b1f-0afc26f251b8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:22.398458 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09bdb9e-4e75-4bb6-9b1f-0afc26f251b8 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:22.423670 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 431/869] 10.222.0.13 () {60 vars in 1141 bytes} [Fri Jun 24 08:28:22 2022] POST /identity/v3/services => generated 251 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:28:22.438323 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:22.483364 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400a3f47-f161-4a05-afa6-73ffc91acd45 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:22.530852 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e55b21-1d69-44f7-a01c-a76a612f73a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:22.531896 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e55b21-1d69-44f7-a01c-a76a612f73a9 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:22.532403 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e55b21-1d69-44f7-a01c-a76a612f73a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:22.532875 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e55b21-1d69-44f7-a01c-a76a612f73a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:22.533854 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e55b21-1d69-44f7-a01c-a76a612f73a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:22.534392 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e55b21-1d69-44f7-a01c-a76a612f73a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:22.536057 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e55b21-1d69-44f7-a01c-a76a612f73a9 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:22.553824 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 439/870] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:28:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:22.557015 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09bdb9e-4e75-4bb6-9b1f-0afc26f251b8 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:22.650771 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:22.668781 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09bdb9e-4e75-4bb6-9b1f-0afc26f251b8 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:22.707877 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326daec9-0b71-4178-9cf6-abb3ae6b9f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:22.708898 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326daec9-0b71-4178-9cf6-abb3ae6b9f4e None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:22.709339 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326daec9-0b71-4178-9cf6-abb3ae6b9f4e None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:22.710009 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326daec9-0b71-4178-9cf6-abb3ae6b9f4e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:22.718356 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326daec9-0b71-4178-9cf6-abb3ae6b9f4e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:22.720152 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326daec9-0b71-4178-9cf6-abb3ae6b9f4e None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:22.735054 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-326daec9-0b71-4178-9cf6-abb3ae6b9f4e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:28:22.735054 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:22.735054 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:22.735054 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:22.735054 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:22.735054 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:22.735054 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:22.735054 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:22.735054 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:22.735054 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:22.735054 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:22.735054 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:22.735054 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:28:22.735054 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:28:22.735054 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:28:22.736536 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:28:22.736536 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:22.736536 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:22.736536 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:22.736536 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:22.736536 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:22.736536 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:22.736536 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:22.736536 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:22.736536 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:22.736536 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:22.736536 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:22.736536 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:22.736536 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:22.736536 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:22.736536 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:22.736536 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:22.737634 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:28:22.737634 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:28:22.737634 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:28:22.737634 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:28:22.737634 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:28:22.737634 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:28:22.737634 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:28:22.737634 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:22.739763 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 432/871] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:28:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:22.747408 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e55b21-1d69-44f7-a01c-a76a612f73a9 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:22.858697 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e55b21-1d69-44f7-a01c-a76a612f73a9 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:22.877674 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:22.897103 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df9271-362a-4c4a-9d5f-b3ef381b02fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:22.897788 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df9271-362a-4c4a-9d5f-b3ef381b02fc None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:22.898075 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df9271-362a-4c4a-9d5f-b3ef381b02fc None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:22.898352 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df9271-362a-4c4a-9d5f-b3ef381b02fc None admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:22.900136 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4df9271-362a-4c4a-9d5f-b3ef381b02fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:22.900470 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4df9271-362a-4c4a-9d5f-b3ef381b02fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:22.901978 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4df9271-362a-4c4a-9d5f-b3ef381b02fc None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:22.920326 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 440/872] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Jun 24 08:28:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:22.926190 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326daec9-0b71-4178-9cf6-abb3ae6b9f4e None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:22.952787 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2df38-8f67-44ea-8640-ab8180798c32 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:22.953070 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2df38-8f67-44ea-8640-ab8180798c32 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:22.953342 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2df38-8f67-44ea-8640-ab8180798c32 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:22.954979 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 441/873] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:28:22.959765 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b713846-33e2-4c1d-886b-512576afa5e6 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:22.959994 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b713846-33e2-4c1d-886b-512576afa5e6 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:22.960267 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b713846-33e2-4c1d-886b-512576afa5e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:22.976786 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-0b713846-33e2-4c1d-886b-512576afa5e6 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:22.978632 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-0b713846-33e2-4c1d-886b-512576afa5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:23.041960 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326daec9-0b71-4178-9cf6-abb3ae6b9f4e None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:23.054002 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 442/874] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:22 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 08:28:23.072678 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b713846-33e2-4c1d-886b-512576afa5e6 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:23.080611 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3930575f-6540-4503-9d4a-3c8b6462b369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:23.081267 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3930575f-6540-4503-9d4a-3c8b6462b369 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:23.081552 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3930575f-6540-4503-9d4a-3c8b6462b369 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:23.081863 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3930575f-6540-4503-9d4a-3c8b6462b369 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:23.082693 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3930575f-6540-4503-9d4a-3c8b6462b369 None admin] RBAC: Adding query filter params (role.id=acfe707eee6944ad94d1915fc9ee7ea3, scope.project.id=180b0af5863349e99ae412b07c1e9465, user.id=bec8c2f113cf41a0939b81f38fbee813) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:23.083005 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3930575f-6540-4503-9d4a-3c8b6462b369 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:23.084505 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3930575f-6540-4503-9d4a-3c8b6462b369 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:23.097866 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 433/875] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Jun 24 08:28:22 2022] GET /identity/v3/role_assignments?user.id=bec8c2f113cf41a0939b81f38fbee813&scope.project.id=180b0af5863349e99ae412b07c1e9465&role.id=acfe707eee6944ad94d1915fc9ee7ea3 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:23.187241 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b713846-33e2-4c1d-886b-512576afa5e6 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:23.199558 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:23.229388 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b84ff-a23a-4c5f-afbc-afe5c50da31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:23.230346 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b84ff-a23a-4c5f-afbc-afe5c50da31d None admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:23.230773 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b84ff-a23a-4c5f-afbc-afe5c50da31d None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:23.231213 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b84ff-a23a-4c5f-afbc-afe5c50da31d None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:23.232106 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1b84ff-a23a-4c5f-afbc-afe5c50da31d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:23.233651 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1b84ff-a23a-4c5f-afbc-afe5c50da31d None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:23.251738 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 443/876] 10.222.0.13 () {60 vars in 1141 bytes} [Fri Jun 24 08:28:23 2022] POST /identity/v3/services => generated 269 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:28:23.739734 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407569a-fc53-40f3-8149-c4e2ff618087 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:23.740268 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407569a-fc53-40f3-8149-c4e2ff618087 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:23.740268 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407569a-fc53-40f3-8149-c4e2ff618087 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:23.741993 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 434/877] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:23.747941 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f4f462-bf36-41c0-9bb9-657fdb210434 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:23.748253 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f4f462-bf36-41c0-9bb9-657fdb210434 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:23.748508 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f4f462-bf36-41c0-9bb9-657fdb210434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:23.766620 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-72f4f462-bf36-41c0-9bb9-657fdb210434 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:23.767813 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-72f4f462-bf36-41c0-9bb9-657fdb210434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:23.846414 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 444/878] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:23 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:28:23.870362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e407569a-fc53-40f3-8149-c4e2ff618087 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:23.885122 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:23.983415 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e407569a-fc53-40f3-8149-c4e2ff618087 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:24.021861 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a373ed9-dbb9-4700-b660-e415545fa308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:24.022600 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a373ed9-dbb9-4700-b660-e415545fa308 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:24.022888 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a373ed9-dbb9-4700-b660-e415545fa308 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:24.023167 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a373ed9-dbb9-4700-b660-e415545fa308 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:24.035131 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a373ed9-dbb9-4700-b660-e415545fa308 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:24.043959 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-5a373ed9-dbb9-4700-b660-e415545fa308 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 08:28:24.043959 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:24.043959 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:24.043959 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:24.043959 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:24.043959 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:24.043959 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:24.043959 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:24.043959 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:24.043959 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:24.043959 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:24.043959 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:24.043959 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:24.043959 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:24.043959 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:24.043959 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:24.044895 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:24.044895 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:24.044895 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:24.044895 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:24.044895 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:24.044895 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:24.044895 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:24.044895 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:24.044895 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:24.044895 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:24.044895 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:24.044895 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:24.044895 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:24.044895 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:24.044895 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:24.044895 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:24.045763 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:24.045763 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:24.045763 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:24.045763 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:24.045763 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:24.045763 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:24.045763 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:24.045763 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:24.045763 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:24.045763 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:24.045763 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 08:28:24.045763 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:24.045763 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 435/879] 10.222.0.13 () {56 vars in 1120 bytes} [Fri Jun 24 08:28:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:24.051120 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f4f462-bf36-41c0-9bb9-657fdb210434 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:24.149953 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:24.157002 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f4f462-bf36-41c0-9bb9-657fdb210434 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:24.195198 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f74e729-5d74-4313-80e5-be354777baa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:24.195981 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f74e729-5d74-4313-80e5-be354777baa0 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:24.196342 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f74e729-5d74-4313-80e5-be354777baa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:24.196624 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f74e729-5d74-4313-80e5-be354777baa0 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:24.197390 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f74e729-5d74-4313-80e5-be354777baa0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:24.197714 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f74e729-5d74-4313-80e5-be354777baa0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:24.199184 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f74e729-5d74-4313-80e5-be354777baa0 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:24.208069 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 445/880] 10.222.0.13 () {56 vars in 1115 bytes} [Fri Jun 24 08:28:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:24.213810 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a373ed9-dbb9-4700-b660-e415545fa308 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:24.289480 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:24.324180 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a373ed9-dbb9-4700-b660-e415545fa308 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:24.363674 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fce14f3-15b7-4663-b67d-76d436a7d671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:24.364474 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce14f3-15b7-4663-b67d-76d436a7d671 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:24.364757 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce14f3-15b7-4663-b67d-76d436a7d671 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:24.365033 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce14f3-15b7-4663-b67d-76d436a7d671 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:24.378180 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fce14f3-15b7-4663-b67d-76d436a7d671 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:24.379767 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fce14f3-15b7-4663-b67d-76d436a7d671 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:24.398923 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-2fce14f3-15b7-4663-b67d-76d436a7d671 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:24.398923 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:24.398923 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:24.398923 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:24.398923 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:24.398923 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:24.398923 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:24.398923 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:24.398923 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:24.398923 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:24.398923 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:24.398923 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:24.398923 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:28:24.398923 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:28:24.398923 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:28:24.400344 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:28:24.400344 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:24.400344 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:24.400344 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:24.400344 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:24.400344 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:24.400344 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:24.400344 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:24.400344 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:24.400344 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:24.400344 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:24.400344 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:24.400344 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:24.400344 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:24.400344 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:24.400344 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:24.400344 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:24.401249 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:28:24.401249 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:28:24.401249 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:24.401249 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:24.401469 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 436/881] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:24.407750 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f74e729-5d74-4313-80e5-be354777baa0 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:24.519540 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f74e729-5d74-4313-80e5-be354777baa0 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:24.558305 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2040b2-1691-4d03-9a74-7d01823f0e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:24.559324 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2040b2-1691-4d03-9a74-7d01823f0e67 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:24.559816 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2040b2-1691-4d03-9a74-7d01823f0e67 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:24.560295 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2040b2-1691-4d03-9a74-7d01823f0e67 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:24.561219 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2040b2-1691-4d03-9a74-7d01823f0e67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:24.561787 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2040b2-1691-4d03-9a74-7d01823f0e67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:24.563359 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2040b2-1691-4d03-9a74-7d01823f0e67 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:24.576952 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 446/882] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:28:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:24.582726 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fce14f3-15b7-4663-b67d-76d436a7d671 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:24.689146 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fce14f3-15b7-4663-b67d-76d436a7d671 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:24.713784 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:24.722292 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b73e84b-9e0a-47e0-a3ef-3ac40532d900 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:24.722893 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b73e84b-9e0a-47e0-a3ef-3ac40532d900 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:24.723567 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b73e84b-9e0a-47e0-a3ef-3ac40532d900 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:24.726194 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 447/883] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:24.732868 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9daf79-3a1e-415d-85a4-00209a2cb6a9 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:24.733241 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9daf79-3a1e-415d-85a4-00209a2cb6a9 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:24.733592 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9daf79-3a1e-415d-85a4-00209a2cb6a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:24.736483 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc32e57-93e2-4f34-898d-eacdfdfecb65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:24.737302 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc32e57-93e2-4f34-898d-eacdfdfecb65 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:24.737586 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc32e57-93e2-4f34-898d-eacdfdfecb65 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:24.737882 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc32e57-93e2-4f34-898d-eacdfdfecb65 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:24.750310 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc32e57-93e2-4f34-898d-eacdfdfecb65 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:24.751860 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc32e57-93e2-4f34-898d-eacdfdfecb65 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:24.757387 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-da9daf79-3a1e-415d-85a4-00209a2cb6a9 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:24.759676 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-da9daf79-3a1e-415d-85a4-00209a2cb6a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:24.769363 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-0fc32e57-93e2-4f34-898d-eacdfdfecb65 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 08:28:24.769363 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:24.769363 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:24.769363 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:24.769363 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:24.769363 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:24.769363 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:24.769363 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:24.769363 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:24.769363 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:24.769363 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:24.769363 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:24.769363 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:28:24.769363 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:28:24.769363 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:28:24.769363 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:28:24.770337 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:24.770337 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:24.770337 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:28:24.770337 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:28:24.770337 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:28:24.770337 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:28:24.770337 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:24.770337 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:24.770337 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:24.770337 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:24.770337 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:24.770337 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:24.770337 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:24.770337 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:24.770337 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:24.770337 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:24.770337 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:24.770337 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:24.771276 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:24.771276 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:24.771276 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:28:24.771276 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:28:24.771276 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:28:24.771276 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:28:24.771276 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:28:24.771276 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:28:24.771276 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 08:28:24.771276 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:24.771801 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 437/884] 10.222.0.13 () {56 vars in 1117 bytes} [Fri Jun 24 08:28:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:24.778767 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc32e57-93e2-4f34-898d-eacdfdfecb65 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:24.780637 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:24.861313 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 448/885] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:24 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:28:24.886096 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9daf79-3a1e-415d-85a4-00209a2cb6a9 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:24.909522 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc32e57-93e2-4f34-898d-eacdfdfecb65 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:24.955285 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e17eb6-e15a-45b5-92d5-355d64155fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:24.955967 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e17eb6-e15a-45b5-92d5-355d64155fe2 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:24.956257 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e17eb6-e15a-45b5-92d5-355d64155fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:24.956542 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e17eb6-e15a-45b5-92d5-355d64155fe2 None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:24.957384 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e17eb6-e15a-45b5-92d5-355d64155fe2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:24.957720 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e17eb6-e15a-45b5-92d5-355d64155fe2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:24.959177 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e17eb6-e15a-45b5-92d5-355d64155fe2 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:24.990879 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 438/886] 10.222.0.13 () {56 vars in 1149 bytes} [Fri Jun 24 08:28:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:24.994168 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e17eb6-e15a-45b5-92d5-355d64155fe2 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:25.024719 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9daf79-3a1e-415d-85a4-00209a2cb6a9 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:25.049594 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:25.077755 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6b31c1-6e80-4476-a130-03bc6d70eaa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:25.079859 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b31c1-6e80-4476-a130-03bc6d70eaa5 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:25.080388 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b31c1-6e80-4476-a130-03bc6d70eaa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:25.080869 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b31c1-6e80-4476-a130-03bc6d70eaa5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:25.081917 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6b31c1-6e80-4476-a130-03bc6d70eaa5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:25.083995 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6b31c1-6e80-4476-a130-03bc6d70eaa5 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:25.100167 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-ad6b31c1-6e80-4476-a130-03bc6d70eaa5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 08:28:25.100167 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:25.100167 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 08:28:25.100167 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 08:28:25.100167 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 08:28:25.100167 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 08:28:25.100167 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 08:28:25.100167 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:28:25.100167 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 08:28:25.100167 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:25.100167 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:28:25.100167 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:25.100167 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:25.100167 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:25.100167 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:25.100167 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:25.100167 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:25.101281 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:25.101281 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:25.101281 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:25.101281 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:25.101281 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:25.101281 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:25.101281 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 08:28:25.101281 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 08:28:25.101281 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 08:28:25.101281 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 08:28:25.101281 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:25.101281 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:25.101281 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:25.101281 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:25.101281 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:25.101281 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:25.102315 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:25.102315 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:25.102315 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:25.102315 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:25.102315 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:25.102315 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:25.102315 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:25.102315 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:25.102315 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:25.102315 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:25.102315 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 08:28:25.102315 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:28:25.102315 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 08:28:25.102315 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:25.110088 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 449/887] 10.222.0.13 () {56 vars in 1114 bytes} [Fri Jun 24 08:28:24 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:28:25.112586 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6b31c1-6e80-4476-a130-03bc6d70eaa5 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:25.127357 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e17eb6-e15a-45b5-92d5-355d64155fe2 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:25.128319 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:25.169632 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9852b3d7-98b5-49e5-987e-b39f5eaaa2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:25.170734 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852b3d7-98b5-49e5-987e-b39f5eaaa2eb None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:25.171226 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852b3d7-98b5-49e5-987e-b39f5eaaa2eb None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:25.171704 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852b3d7-98b5-49e5-987e-b39f5eaaa2eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:25.226933 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6b31c1-6e80-4476-a130-03bc6d70eaa5 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:25.252385 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9852b3d7-98b5-49e5-987e-b39f5eaaa2eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:25.254278 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9852b3d7-98b5-49e5-987e-b39f5eaaa2eb None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:25.264735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a0129f-3d29-4645-81a8-77610dde57fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:25.265777 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0129f-3d29-4645-81a8-77610dde57fe None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:25.265894 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0129f-3d29-4645-81a8-77610dde57fe None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:25.266088 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0129f-3d29-4645-81a8-77610dde57fe None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:25.266803 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a0129f-3d29-4645-81a8-77610dde57fe None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:25.267110 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a0129f-3d29-4645-81a8-77610dde57fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:25.268579 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a0129f-3d29-4645-81a8-77610dde57fe None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:25.278121 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 450/888] 10.222.0.13 () {56 vars in 1114 bytes} [Fri Jun 24 08:28:25 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:25.280472 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a0129f-3d29-4645-81a8-77610dde57fe None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:25.304020 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:25.371733 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:25.381878 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 439/889] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:28:24 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 08:28:25.381878 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df92b2-84a7-47b6-90d6-6fb37884c20c None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:25.381878 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df92b2-84a7-47b6-90d6-6fb37884c20c None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:25.381878 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df92b2-84a7-47b6-90d6-6fb37884c20c None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:25.384680 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 440/890] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:25.386024 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df92b2-84a7-47b6-90d6-6fb37884c20c None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:25.404516 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a0129f-3d29-4645-81a8-77610dde57fe None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:25.444901 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cdb20b-b1d1-4a6a-a1b1-a3da22db4fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:25.445582 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cdb20b-b1d1-4a6a-a1b1-a3da22db4fdc None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:25.446564 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cdb20b-b1d1-4a6a-a1b1-a3da22db4fdc None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:25.447148 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cdb20b-b1d1-4a6a-a1b1-a3da22db4fdc None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:25.447542 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cdb20b-b1d1-4a6a-a1b1-a3da22db4fdc None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:25.447849 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cdb20b-b1d1-4a6a-a1b1-a3da22db4fdc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:25.449212 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cdb20b-b1d1-4a6a-a1b1-a3da22db4fdc None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:25.454006 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 451/891] 10.222.0.13 () {56 vars in 1114 bytes} [Fri Jun 24 08:28:25 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:25.455230 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdd8f7b-98ad-4130-8776-c6cf903c80d1 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:25.455528 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdd8f7b-98ad-4130-8776-c6cf903c80d1 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:25.455810 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdd8f7b-98ad-4130-8776-c6cf903c80d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:25.472495 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-4bdd8f7b-98ad-4130-8776-c6cf903c80d1 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:25.473711 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-4bdd8f7b-98ad-4130-8776-c6cf903c80d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:25.498179 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df92b2-84a7-47b6-90d6-6fb37884c20c None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:25.534559 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0159ef5-90e6-4142-bc89-79f048c6b29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:25.535346 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0159ef5-90e6-4142-bc89-79f048c6b29f None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:25.535633 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0159ef5-90e6-4142-bc89-79f048c6b29f None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:25.535912 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0159ef5-90e6-4142-bc89-79f048c6b29f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:25.542370 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 452/892] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:25 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:28:25.543705 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e973f-1062-4485-869a-7080ed81278e None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:25.544008 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e973f-1062-4485-869a-7080ed81278e None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:25.544267 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e973f-1062-4485-869a-7080ed81278e None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:25.545479 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:25.545591 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 453/893] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:25.546246 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845e973f-1062-4485-869a-7080ed81278e None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:25.548330 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0159ef5-90e6-4142-bc89-79f048c6b29f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:25.549981 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0159ef5-90e6-4142-bc89-79f048c6b29f None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:25.568239 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-f0159ef5-90e6-4142-bc89-79f048c6b29f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:25.568239 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:25.568239 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:25.568239 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:25.568239 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:25.568239 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:25.568239 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:25.568239 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:25.568239 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:25.568239 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:25.568239 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:25.568239 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:25.568239 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:28:25.568239 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:28:25.568239 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:28:25.569144 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:28:25.569144 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:25.569144 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:25.569144 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:25.569144 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:25.569144 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:25.569144 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:25.569144 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:25.569144 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:25.569144 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:25.569144 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:25.569144 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:25.569144 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:25.569144 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:25.569144 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:25.569144 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:25.569144 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:25.570071 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:28:25.570071 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:28:25.570071 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:25.570071 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:25.570071 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 441/894] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:25.570665 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412065e7-4cdd-4bdf-a87f-d9eee3d619da None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:25.571856 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412065e7-4cdd-4bdf-a87f-d9eee3d619da None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:25.571974 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412065e7-4cdd-4bdf-a87f-d9eee3d619da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:25.589730 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-412065e7-4cdd-4bdf-a87f-d9eee3d619da None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:25.590930 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-412065e7-4cdd-4bdf-a87f-d9eee3d619da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:25.625179 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:25.657983 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 442/895] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:25 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:28:25.658889 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412065e7-4cdd-4bdf-a87f-d9eee3d619da None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:25.661198 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845e973f-1062-4485-869a-7080ed81278e None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:25.699742 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d954c9-0739-48dc-b2b9-ad8b3d0a69ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:25.700395 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d954c9-0739-48dc-b2b9-ad8b3d0a69ca None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:25.700675 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d954c9-0739-48dc-b2b9-ad8b3d0a69ca None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:25.700999 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d954c9-0739-48dc-b2b9-ad8b3d0a69ca None admin] PATH_INFO: `/v3/endpoints` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:25.701758 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d954c9-0739-48dc-b2b9-ad8b3d0a69ca None admin] RBAC: Adding query filter params (interface=public, service_id=08d889d83acf457c9adc57d7f27a43ba, region_id=RegionOne) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:25.702064 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d954c9-0739-48dc-b2b9-ad8b3d0a69ca None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:25.703415 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d954c9-0739-48dc-b2b9-ad8b3d0a69ca None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:25.708428 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 454/896] 10.222.0.13 () {56 vars in 1251 bytes} [Fri Jun 24 08:28:25 2022] GET /identity/v3/endpoints?service_id=08d889d83acf457c9adc57d7f27a43ba&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:25.709179 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d954c9-0739-48dc-b2b9-ad8b3d0a69ca None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:25.760131 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412065e7-4cdd-4bdf-a87f-d9eee3d619da None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:25.783912 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:25.796049 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56644774-489f-4ec8-8641-5243fa268005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:25.796849 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56644774-489f-4ec8-8641-5243fa268005 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:25.797134 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56644774-489f-4ec8-8641-5243fa268005 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:25.797407 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56644774-489f-4ec8-8641-5243fa268005 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:25.798084 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56644774-489f-4ec8-8641-5243fa268005 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:25.799448 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56644774-489f-4ec8-8641-5243fa268005 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:25.811132 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-56644774-489f-4ec8-8641-5243fa268005 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 08:28:25.811132 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:25.811132 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 08:28:25.811132 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 08:28:25.811132 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 08:28:25.811132 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 08:28:25.811132 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 08:28:25.811132 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:28:25.811132 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 08:28:25.811132 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:25.811132 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:28:25.811132 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:25.811132 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:25.811132 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:25.811132 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:25.811132 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:25.811132 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:25.812112 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:25.812112 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:25.812112 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:25.812112 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:25.812112 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:25.812112 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:25.812112 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 08:28:25.812112 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 08:28:25.812112 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 08:28:25.812112 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 08:28:25.812112 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:25.812112 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:25.812112 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:25.812112 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:25.812112 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:25.812112 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:25.813124 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:25.813124 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:25.813124 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:25.813124 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:25.813124 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:25.813124 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:25.813124 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:25.813124 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:25.813124 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:25.813124 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:25.813124 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 08:28:25.813124 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:28:25.813124 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 08:28:25.813124 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:25.813124 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 443/897] 10.222.0.13 () {56 vars in 1126 bytes} [Fri Jun 24 08:28:25 2022] GET /identity/v3/services/object-store => generated 93 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:25.814178 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56644774-489f-4ec8-8641-5243fa268005 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:25.819060 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d954c9-0739-48dc-b2b9-ad8b3d0a69ca None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:25.849866 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:25.855181 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6205ad8b-5145-4a8d-b7aa-c705a10d4270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:25.855836 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6205ad8b-5145-4a8d-b7aa-c705a10d4270 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:25.856118 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6205ad8b-5145-4a8d-b7aa-c705a10d4270 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:25.856397 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6205ad8b-5145-4a8d-b7aa-c705a10d4270 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:25.857119 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6205ad8b-5145-4a8d-b7aa-c705a10d4270 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:25.857422 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6205ad8b-5145-4a8d-b7aa-c705a10d4270 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:25.858816 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6205ad8b-5145-4a8d-b7aa-c705a10d4270 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:25.870845 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 455/898] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:28:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:25.871625 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6205ad8b-5145-4a8d-b7aa-c705a10d4270 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:25.915987 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56644774-489f-4ec8-8641-5243fa268005 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:25.952582 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432b5234-2f78-4080-8327-61202f3961b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:25.953384 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b5234-2f78-4080-8327-61202f3961b4 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:25.953662 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b5234-2f78-4080-8327-61202f3961b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:25.953965 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b5234-2f78-4080-8327-61202f3961b4 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:25.954604 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432b5234-2f78-4080-8327-61202f3961b4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:25.955956 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432b5234-2f78-4080-8327-61202f3961b4 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:25.967794 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-432b5234-2f78-4080-8327-61202f3961b4 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 08:28:25.967794 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:25.967794 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 08:28:25.967794 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 08:28:25.967794 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 08:28:25.967794 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 08:28:25.967794 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 08:28:25.967794 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:28:25.967794 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 08:28:25.967794 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:25.967794 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:28:25.967794 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:25.967794 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:25.967794 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:25.967794 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:25.967794 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:25.968732 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:25.968732 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:25.968732 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:25.968732 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:25.968732 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:25.968732 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:25.968732 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:25.968732 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 08:28:25.968732 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 08:28:25.968732 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 08:28:25.968732 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 08:28:25.968732 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:25.968732 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:25.968732 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:25.968732 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:25.968732 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:25.969590 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:25.969590 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:25.969590 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:25.969590 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:25.969590 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:25.969590 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:25.969590 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:25.969590 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:25.969590 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:25.969590 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:25.969590 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:25.969590 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 08:28:25.969590 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:28:25.969590 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 08:28:25.969590 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:25.969590 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 444/899] 10.222.0.13 () {56 vars in 1132 bytes} [Fri Jun 24 08:28:25 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:25.970470 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432b5234-2f78-4080-8327-61202f3961b4 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:25.979242 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6205ad8b-5145-4a8d-b7aa-c705a10d4270 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:26.014220 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcef625-ec24-45d4-bfda-82bbb6892963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:26.014883 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcef625-ec24-45d4-bfda-82bbb6892963 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:26.015168 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcef625-ec24-45d4-bfda-82bbb6892963 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:26.015446 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcef625-ec24-45d4-bfda-82bbb6892963 None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:26.016122 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcef625-ec24-45d4-bfda-82bbb6892963 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:26.017481 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcef625-ec24-45d4-bfda-82bbb6892963 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:26.020834 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:26.023401 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 456/900] 10.222.0.13 () {56 vars in 1087 bytes} [Fri Jun 24 08:28:25 2022] GET /identity/v3/services => generated 1268 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:28:26.023830 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcef625-ec24-45d4-bfda-82bbb6892963 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:26.082710 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432b5234-2f78-4080-8327-61202f3961b4 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:26.090191 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:26.122689 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb72a0-9fbc-4a51-9b13-093ca5399acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:26.123369 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb72a0-9fbc-4a51-9b13-093ca5399acc None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:26.123659 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb72a0-9fbc-4a51-9b13-093ca5399acc None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:26.124004 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb72a0-9fbc-4a51-9b13-093ca5399acc None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:26.124660 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccb72a0-9fbc-4a51-9b13-093ca5399acc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:26.124971 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccb72a0-9fbc-4a51-9b13-093ca5399acc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:26.126417 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccb72a0-9fbc-4a51-9b13-093ca5399acc None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:26.132059 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 445/901] 10.222.0.13 () {56 vars in 1122 bytes} [Fri Jun 24 08:28:25 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:26.132922 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb72a0-9fbc-4a51-9b13-093ca5399acc None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:26.154338 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcef625-ec24-45d4-bfda-82bbb6892963 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:26.196545 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb6370d-4b27-42a3-9914-90cd914cbbdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:26.197494 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb6370d-4b27-42a3-9914-90cd914cbbdf None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:26.197841 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb6370d-4b27-42a3-9914-90cd914cbbdf None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:26.198126 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb6370d-4b27-42a3-9914-90cd914cbbdf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:26.205960 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb6370d-4b27-42a3-9914-90cd914cbbdf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:26.207410 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb6370d-4b27-42a3-9914-90cd914cbbdf None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:26.222056 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-deb6370d-4b27-42a3-9914-90cd914cbbdf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:28:26.222056 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:26.222056 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:26.222056 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:26.222056 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:26.222056 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:26.222056 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:26.222056 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:26.222056 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:26.222056 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:26.222056 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:26.222056 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:26.222056 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:28:26.222056 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:28:26.222056 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:28:26.222994 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:28:26.222994 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:26.222994 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:26.222994 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:26.222994 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:26.222994 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:26.222994 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:26.222994 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:26.222994 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:26.222994 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:26.222994 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:26.222994 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:26.222994 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:26.222994 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:26.222994 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:26.222994 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:26.222994 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:26.223888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:28:26.223888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:28:26.223888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:28:26.223888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:28:26.223888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:28:26.223888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:28:26.223888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:28:26.223888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:26.223888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 457/902] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:28:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:26.224367 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb6370d-4b27-42a3-9914-90cd914cbbdf None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:26.251659 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb72a0-9fbc-4a51-9b13-093ca5399acc None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:26.266470 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:26.288038 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca15c228-2126-4e4d-b344-42574644c3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:26.288699 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca15c228-2126-4e4d-b344-42574644c3ad None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:26.288980 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca15c228-2126-4e4d-b344-42574644c3ad None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:26.289257 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca15c228-2126-4e4d-b344-42574644c3ad None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:26.290950 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca15c228-2126-4e4d-b344-42574644c3ad None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:26.291256 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca15c228-2126-4e4d-b344-42574644c3ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:26.292635 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca15c228-2126-4e4d-b344-42574644c3ad None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:26.297508 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 446/903] 10.222.0.13 () {56 vars in 1126 bytes} [Fri Jun 24 08:28:26 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:26.298309 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca15c228-2126-4e4d-b344-42574644c3ad None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:26.345748 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb6370d-4b27-42a3-9914-90cd914cbbdf None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:26.365781 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:26.389008 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1783b29-a72e-4702-aa36-611426876221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:26.389670 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1783b29-a72e-4702-aa36-611426876221 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:26.390203 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1783b29-a72e-4702-aa36-611426876221 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:26.390342 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1783b29-a72e-4702-aa36-611426876221 None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:26.390970 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1783b29-a72e-4702-aa36-611426876221 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:26.391526 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1783b29-a72e-4702-aa36-611426876221 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:26.392692 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1783b29-a72e-4702-aa36-611426876221 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:26.397370 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 458/904] 10.222.0.13 () {56 vars in 1122 bytes} [Fri Jun 24 08:28:26 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:26.406128 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1783b29-a72e-4702-aa36-611426876221 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:26.424233 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca15c228-2126-4e4d-b344-42574644c3ad None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:26.469768 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de61874-d856-4422-a289-b8533d80e1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:26.470554 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de61874-d856-4422-a289-b8533d80e1ce None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:26.470840 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de61874-d856-4422-a289-b8533d80e1ce None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:26.471530 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de61874-d856-4422-a289-b8533d80e1ce None admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:26.472628 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de61874-d856-4422-a289-b8533d80e1ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:26.472938 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de61874-d856-4422-a289-b8533d80e1ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:26.475177 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de61874-d856-4422-a289-b8533d80e1ce None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:26.493549 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 447/905] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Jun 24 08:28:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:26.500027 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de61874-d856-4422-a289-b8533d80e1ce None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:26.540563 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1783b29-a72e-4702-aa36-611426876221 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:26.554216 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:26.593226 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e895155b-cfea-416d-bfa4-c3a49cc69670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:26.594193 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895155b-cfea-416d-bfa4-c3a49cc69670 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:26.594641 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895155b-cfea-416d-bfa4-c3a49cc69670 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:26.595068 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895155b-cfea-416d-bfa4-c3a49cc69670 None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:26.595781 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e895155b-cfea-416d-bfa4-c3a49cc69670 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:26.596086 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e895155b-cfea-416d-bfa4-c3a49cc69670 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:26.597459 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e895155b-cfea-416d-bfa4-c3a49cc69670 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:26.609715 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 459/906] 10.222.0.13 () {56 vars in 1126 bytes} [Fri Jun 24 08:28:26 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:26.611858 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e895155b-cfea-416d-bfa4-c3a49cc69670 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:26.637330 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de61874-d856-4422-a289-b8533d80e1ce None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:26.672478 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9794f6e6-7412-4919-a2aa-9a4c55f73c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:26.673135 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9794f6e6-7412-4919-a2aa-9a4c55f73c56 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:26.673417 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9794f6e6-7412-4919-a2aa-9a4c55f73c56 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:26.673720 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9794f6e6-7412-4919-a2aa-9a4c55f73c56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:26.675084 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9794f6e6-7412-4919-a2aa-9a4c55f73c56 None admin] RBAC: Adding query filter params (role.id=f9658866044e4fdeaddcc91781166faf, scope.project.id=180b0af5863349e99ae412b07c1e9465, user.id=85d9c4c57681447c898ebfd7990b05b6) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:26.675579 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9794f6e6-7412-4919-a2aa-9a4c55f73c56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:26.676137 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:26.677186 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9794f6e6-7412-4919-a2aa-9a4c55f73c56 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:26.684171 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 448/907] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Jun 24 08:28:26 2022] GET /identity/v3/role_assignments?user.id=85d9c4c57681447c898ebfd7990b05b6&scope.project.id=180b0af5863349e99ae412b07c1e9465&role.id=f9658866044e4fdeaddcc91781166faf => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:26.746113 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e895155b-cfea-416d-bfa4-c3a49cc69670 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:26.787234 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1b65e7-0d8e-48cc-999c-a2e4a8fa05e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:26.788198 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1b65e7-0d8e-48cc-999c-a2e4a8fa05e9 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:26.788656 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1b65e7-0d8e-48cc-999c-a2e4a8fa05e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:26.789087 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1b65e7-0d8e-48cc-999c-a2e4a8fa05e9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:26.790627 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1b65e7-0d8e-48cc-999c-a2e4a8fa05e9 None admin] RBAC: Adding query filter params (interface=public, service_id=12a8be8868634ca78e53657c095aaf98, region_id=RegionOne) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:26.791174 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1b65e7-0d8e-48cc-999c-a2e4a8fa05e9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:26.792732 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1b65e7-0d8e-48cc-999c-a2e4a8fa05e9 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:26.803569 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 460/908] 10.222.0.13 () {56 vars in 1251 bytes} [Fri Jun 24 08:28:26 2022] GET /identity/v3/endpoints?service_id=12a8be8868634ca78e53657c095aaf98&interface=public®ion_id=RegionOne => generated 194 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:26.805405 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9794f6e6-7412-4919-a2aa-9a4c55f73c56 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:26.926707 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9794f6e6-7412-4919-a2aa-9a4c55f73c56 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:26.928759 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:26.963507 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e468e1-4506-4df5-a11e-e2431ba0258c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:26.964175 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e468e1-4506-4df5-a11e-e2431ba0258c None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:26.964461 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e468e1-4506-4df5-a11e-e2431ba0258c None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:26.964737 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e468e1-4506-4df5-a11e-e2431ba0258c None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:26.965414 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e468e1-4506-4df5-a11e-e2431ba0258c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:26.966837 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e468e1-4506-4df5-a11e-e2431ba0258c None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:26.977830 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 449/909] 10.222.0.13 () {56 vars in 1087 bytes} [Fri Jun 24 08:28:26 2022] GET /identity/v3/services => generated 1268 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:28:28.128846 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65613826-1fbd-41db-a442-71ce1406c687 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:28.129403 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65613826-1fbd-41db-a442-71ce1406c687 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:28.129403 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65613826-1fbd-41db-a442-71ce1406c687 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:28.131031 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 461/910] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:28.137093 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da432119-fecf-4c8c-a17e-0ce0c3b97366 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:28.137532 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da432119-fecf-4c8c-a17e-0ce0c3b97366 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:28.137760 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da432119-fecf-4c8c-a17e-0ce0c3b97366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:28.154523 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-da432119-fecf-4c8c-a17e-0ce0c3b97366 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:28.155720 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-da432119-fecf-4c8c-a17e-0ce0c3b97366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:28.223294 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 450/911] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:28 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 08:28:28.246736 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65613826-1fbd-41db-a442-71ce1406c687 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:28.361868 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65613826-1fbd-41db-a442-71ce1406c687 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:28.377727 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:28.408543 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12d98cd-58bf-41f5-8702-69470a0d50ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:28.409595 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d98cd-58bf-41f5-8702-69470a0d50ef None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:28.410060 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d98cd-58bf-41f5-8702-69470a0d50ef None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:28.410584 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d98cd-58bf-41f5-8702-69470a0d50ef None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:28.411385 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12d98cd-58bf-41f5-8702-69470a0d50ef None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:28.412895 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12d98cd-58bf-41f5-8702-69470a0d50ef None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:28.426114 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-d12d98cd-58bf-41f5-8702-69470a0d50ef None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 08:28:28.426114 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:28.426114 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 08:28:28.426114 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 08:28:28.426114 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 08:28:28.426114 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 08:28:28.426114 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 08:28:28.426114 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:28:28.426114 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 08:28:28.426114 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:28.426114 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:28:28.426114 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:28.426114 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:28.426114 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:28.426114 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:28.426114 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:28.426114 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:28.427412 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:28.427412 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:28.427412 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:28.427412 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:28.427412 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:28.427412 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:28.427412 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 08:28:28.427412 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 08:28:28.427412 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 08:28:28.427412 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 08:28:28.427412 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:28.427412 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:28.427412 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:28.427412 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:28.427412 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:28.427412 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:28.428503 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:28.428503 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:28.428503 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:28.428503 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:28.428503 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:28.428503 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:28.428503 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:28.428503 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:28.428503 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:28.428503 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:28.428503 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 08:28:28.428503 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:28:28.428503 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 08:28:28.428503 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:28.431759 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 462/912] 10.222.0.13 () {56 vars in 1114 bytes} [Fri Jun 24 08:28:28 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:28.437540 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da432119-fecf-4c8c-a17e-0ce0c3b97366 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:28.540360 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:28.547290 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da432119-fecf-4c8c-a17e-0ce0c3b97366 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:28.580399 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8c39b-ccab-4beb-aabd-0712383c63d3 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:28.580686 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8c39b-ccab-4beb-aabd-0712383c63d3 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:28.580972 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8c39b-ccab-4beb-aabd-0712383c63d3 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:28.582559 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 463/913] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:28:28.583966 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001f9f58-6770-4d41-84be-4d08a903f00d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:28.584923 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f9f58-6770-4d41-84be-4d08a903f00d None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:28.585403 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f9f58-6770-4d41-84be-4d08a903f00d None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:28.585901 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f9f58-6770-4d41-84be-4d08a903f00d None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:28.587404 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001f9f58-6770-4d41-84be-4d08a903f00d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:28.587947 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001f9f58-6770-4d41-84be-4d08a903f00d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:28.588698 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8988bdc8-cb86-46ad-8009-7932aea6bc50 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:28.588990 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8988bdc8-cb86-46ad-8009-7932aea6bc50 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:28.589274 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8988bdc8-cb86-46ad-8009-7932aea6bc50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:28.590205 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001f9f58-6770-4d41-84be-4d08a903f00d None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:28.595703 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 451/914] 10.222.0.13 () {56 vars in 1114 bytes} [Fri Jun 24 08:28:28 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:28.601330 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001f9f58-6770-4d41-84be-4d08a903f00d None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:28.609020 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-8988bdc8-cb86-46ad-8009-7932aea6bc50 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:28.610264 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-8988bdc8-cb86-46ad-8009-7932aea6bc50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:28.694025 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 464/915] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:28 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 08:28:28.709683 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8988bdc8-cb86-46ad-8009-7932aea6bc50 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:28.711049 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001f9f58-6770-4d41-84be-4d08a903f00d None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:28.754752 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feda0637-7dec-4ae1-a63f-8434a3a830dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:28.755426 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda0637-7dec-4ae1-a63f-8434a3a830dd None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:28.755710 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda0637-7dec-4ae1-a63f-8434a3a830dd None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:28.755985 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda0637-7dec-4ae1-a63f-8434a3a830dd None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:28.756750 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feda0637-7dec-4ae1-a63f-8434a3a830dd None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:28.757060 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feda0637-7dec-4ae1-a63f-8434a3a830dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:28.758814 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feda0637-7dec-4ae1-a63f-8434a3a830dd None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:28.768679 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 452/916] 10.222.0.13 () {56 vars in 1114 bytes} [Fri Jun 24 08:28:28 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:28.770241 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feda0637-7dec-4ae1-a63f-8434a3a830dd None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:28.797875 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:28.829221 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8988bdc8-cb86-46ad-8009-7932aea6bc50 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:28.831213 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:28.871775 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe465046-f433-4a0e-955c-f5a9876deaa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:28.872451 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe465046-f433-4a0e-955c-f5a9876deaa1 None admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:28.872738 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe465046-f433-4a0e-955c-f5a9876deaa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:28.873016 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe465046-f433-4a0e-955c-f5a9876deaa1 None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:28.873769 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe465046-f433-4a0e-955c-f5a9876deaa1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:28.875146 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe465046-f433-4a0e-955c-f5a9876deaa1 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:28.889185 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feda0637-7dec-4ae1-a63f-8434a3a830dd None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:28.891415 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 465/917] 10.222.0.13 () {60 vars in 1141 bytes} [Fri Jun 24 08:28:28 2022] POST /identity/v3/services => generated 241 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:28:28.900355 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce939b9-54d6-484f-97e9-fcba65bd73ee None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:28.900792 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce939b9-54d6-484f-97e9-fcba65bd73ee None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:28.901227 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce939b9-54d6-484f-97e9-fcba65bd73ee None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:28.908232 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 466/918] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:28.910597 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f31cd8-0aa9-471a-a0a5-aa1d4e37b1b0 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:28.910923 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f31cd8-0aa9-471a-a0a5-aa1d4e37b1b0 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:28.911256 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f31cd8-0aa9-471a-a0a5-aa1d4e37b1b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:28.937150 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-13f31cd8-0aa9-471a-a0a5-aa1d4e37b1b0 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:28.939155 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-13f31cd8-0aa9-471a-a0a5-aa1d4e37b1b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:28.953427 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae840412-e363-48f4-a972-d449819079c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:28.954477 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae840412-e363-48f4-a972-d449819079c0 None admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:28.954976 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae840412-e363-48f4-a972-d449819079c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:28.955460 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae840412-e363-48f4-a972-d449819079c0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:28.956444 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae840412-e363-48f4-a972-d449819079c0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:28.958173 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae840412-e363-48f4-a972-d449819079c0 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:28.988344 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 453/919] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Jun 24 08:28:28 2022] POST /identity/v3/endpoints => generated 346 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:28:29.048892 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 467/920] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:28:29.073924 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae840412-e363-48f4-a972-d449819079c0 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:29.145302 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6235ba-ccc9-400a-bbf4-42fd3f738fb6 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:29.146306 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6235ba-ccc9-400a-bbf4-42fd3f738fb6 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:29.146799 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6235ba-ccc9-400a-bbf4-42fd3f738fb6 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:29.148705 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 468/921] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:29.154563 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f278e2-e874-4511-8711-b5a56040f65b None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:29.155449 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f278e2-e874-4511-8711-b5a56040f65b None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:29.156319 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f278e2-e874-4511-8711-b5a56040f65b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:29.174663 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:29.179964 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-b6f278e2-e874-4511-8711-b5a56040f65b None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:29.181437 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b6f278e2-e874-4511-8711-b5a56040f65b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:29.206744 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae840412-e363-48f4-a972-d449819079c0 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:29.246289 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87971a71-dac2-49bf-a626-43dac3679b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:29.246992 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87971a71-dac2-49bf-a626-43dac3679b7a None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:29.247280 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87971a71-dac2-49bf-a626-43dac3679b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:29.247557 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87971a71-dac2-49bf-a626-43dac3679b7a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:29.259586 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87971a71-dac2-49bf-a626-43dac3679b7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:29.267812 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-87971a71-dac2-49bf-a626-43dac3679b7a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 08:28:29.267812 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:29.267812 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:29.267812 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:29.267812 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:29.267812 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:29.267812 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:29.267812 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:29.267812 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:29.267812 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:29.267812 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:29.267812 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:29.267812 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:29.267812 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:29.267812 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:29.267812 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:29.268748 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:29.268748 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:29.268748 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:29.268748 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:29.268748 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:29.268748 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:29.268748 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:29.268748 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:29.268748 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:29.268748 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:29.268748 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:29.268748 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:29.268748 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:29.268748 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:29.268748 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:29.268748 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:29.269577 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:29.269577 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:29.269577 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:29.269577 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:29.269577 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:29.269577 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:29.269577 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:29.269577 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:29.269577 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:29.269577 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:29.269577 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 08:28:29.269577 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:29.269577 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 454/922] 10.222.0.13 () {56 vars in 1120 bytes} [Fri Jun 24 08:28:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:29.274979 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 469/923] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 08:28:29.281357 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87971a71-dac2-49bf-a626-43dac3679b7a None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:29.308961 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f278e2-e874-4511-8711-b5a56040f65b None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:29.330839 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:29.407923 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87971a71-dac2-49bf-a626-43dac3679b7a None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:29.434399 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f278e2-e874-4511-8711-b5a56040f65b None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:29.444562 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce77967e-a9c1-4cf4-840e-013caf2bf608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:29.445451 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77967e-a9c1-4cf4-840e-013caf2bf608 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:29.446453 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77967e-a9c1-4cf4-840e-013caf2bf608 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:29.446899 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77967e-a9c1-4cf4-840e-013caf2bf608 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:29.447810 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce77967e-a9c1-4cf4-840e-013caf2bf608 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:29.448288 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce77967e-a9c1-4cf4-840e-013caf2bf608 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:29.449843 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce77967e-a9c1-4cf4-840e-013caf2bf608 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:29.452567 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:29.464715 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 455/924] 10.222.0.13 () {56 vars in 1115 bytes} [Fri Jun 24 08:28:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:29.465942 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce77967e-a9c1-4cf4-840e-013caf2bf608 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:29.474980 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76b6418-a407-4986-956a-e386cb3c4830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:29.475902 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b6418-a407-4986-956a-e386cb3c4830 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:29.476071 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b6418-a407-4986-956a-e386cb3c4830 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:29.476348 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b6418-a407-4986-956a-e386cb3c4830 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:29.477007 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76b6418-a407-4986-956a-e386cb3c4830 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:29.479231 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76b6418-a407-4986-956a-e386cb3c4830 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:29.492166 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-e76b6418-a407-4986-956a-e386cb3c4830 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 08:28:29.492166 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:29.492166 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 08:28:29.492166 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 08:28:29.492166 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 08:28:29.492166 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 08:28:29.492166 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 08:28:29.492166 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:28:29.492166 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 08:28:29.492166 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:29.492166 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:28:29.492166 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:29.492166 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:29.492166 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:29.492166 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:29.492166 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:29.493217 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:29.493217 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:29.493217 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:29.493217 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:29.493217 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:29.493217 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:29.493217 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:29.493217 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 08:28:29.493217 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 08:28:29.493217 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 08:28:29.493217 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 08:28:29.493217 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:29.493217 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:29.493217 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:29.493217 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:29.493217 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:29.494126 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:29.494126 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:29.494126 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:29.494126 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:29.494126 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:29.494126 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:29.494126 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:29.494126 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:29.494126 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:29.494126 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:29.494126 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:29.494126 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 08:28:29.494126 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:28:29.494126 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 08:28:29.494126 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:29.499052 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 470/925] 10.222.0.13 () {56 vars in 1132 bytes} [Fri Jun 24 08:28:29 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:29.500137 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76b6418-a407-4986-956a-e386cb3c4830 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:29.590044 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce77967e-a9c1-4cf4-840e-013caf2bf608 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:29.594103 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:29.615414 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76b6418-a407-4986-956a-e386cb3c4830 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:29.636827 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccb45f0-469b-4d60-affe-489fee8e522f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:29.637847 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccb45f0-469b-4d60-affe-489fee8e522f None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:29.638202 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccb45f0-469b-4d60-affe-489fee8e522f None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:29.638549 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccb45f0-469b-4d60-affe-489fee8e522f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:29.650710 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccb45f0-469b-4d60-affe-489fee8e522f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:29.652299 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccb45f0-469b-4d60-affe-489fee8e522f None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:29.656304 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55506e17-b766-4ad8-8317-af2ae338051d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:29.657144 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55506e17-b766-4ad8-8317-af2ae338051d None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:29.657569 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55506e17-b766-4ad8-8317-af2ae338051d None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:29.658033 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55506e17-b766-4ad8-8317-af2ae338051d None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:29.658879 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55506e17-b766-4ad8-8317-af2ae338051d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:29.659348 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55506e17-b766-4ad8-8317-af2ae338051d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:29.661417 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55506e17-b766-4ad8-8317-af2ae338051d None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:29.667794 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 471/926] 10.222.0.13 () {56 vars in 1126 bytes} [Fri Jun 24 08:28:29 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:29.674147 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-3ccb45f0-469b-4d60-affe-489fee8e522f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:29.674147 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:29.674147 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:29.674147 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:29.674147 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:29.674147 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:29.674147 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:29.674147 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:29.674147 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:29.674147 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:29.674147 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:29.674147 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:29.674147 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:28:29.674147 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:28:29.674147 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:28:29.675026 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:28:29.675026 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:29.675026 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:29.675026 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:29.675026 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:29.675026 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:29.675026 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:29.675026 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:29.675026 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:29.675026 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:29.675026 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:29.675026 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:29.675026 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:29.675026 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:29.675026 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:29.675026 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:29.675026 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:29.675907 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:28:29.675907 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:28:29.675907 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:29.675907 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:29.676144 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55506e17-b766-4ad8-8317-af2ae338051d None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:29.677819 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 456/927] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:29.686701 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccb45f0-469b-4d60-affe-489fee8e522f None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:29.750848 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:29.807443 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55506e17-b766-4ad8-8317-af2ae338051d None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:29.814241 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccb45f0-469b-4d60-affe-489fee8e522f None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:29.847423 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fcb5f9-289f-4607-946f-affe8c9fb423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:29.848189 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fcb5f9-289f-4607-946f-affe8c9fb423 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:29.848543 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fcb5f9-289f-4607-946f-affe8c9fb423 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:29.849006 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fcb5f9-289f-4607-946f-affe8c9fb423 None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:29.849821 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fcb5f9-289f-4607-946f-affe8c9fb423 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:29.850274 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fcb5f9-289f-4607-946f-affe8c9fb423 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:29.851774 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fcb5f9-289f-4607-946f-affe8c9fb423 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:29.860033 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0590416-8a45-440b-82b5-31e1f0fa2f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:29.860935 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0590416-8a45-440b-82b5-31e1f0fa2f7d None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:29.861379 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0590416-8a45-440b-82b5-31e1f0fa2f7d None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:29.861850 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0590416-8a45-440b-82b5-31e1f0fa2f7d None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:29.862743 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0590416-8a45-440b-82b5-31e1f0fa2f7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:29.863225 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0590416-8a45-440b-82b5-31e1f0fa2f7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:29.863338 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 472/928] 10.222.0.13 () {56 vars in 1126 bytes} [Fri Jun 24 08:28:29 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:29.864154 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fcb5f9-289f-4607-946f-affe8c9fb423 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:29.865173 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0590416-8a45-440b-82b5-31e1f0fa2f7d None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:29.878396 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:29.884923 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 457/929] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:28:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:29.887273 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0590416-8a45-440b-82b5-31e1f0fa2f7d None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:29.992589 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fcb5f9-289f-4607-946f-affe8c9fb423 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:30.012606 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0590416-8a45-440b-82b5-31e1f0fa2f7d None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:30.024784 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:30.039629 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2b8dae-605b-4751-8eab-4de5487ba784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:30.041191 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2b8dae-605b-4751-8eab-4de5487ba784 None admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:30.041929 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2b8dae-605b-4751-8eab-4de5487ba784 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:30.042382 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2b8dae-605b-4751-8eab-4de5487ba784 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:30.043546 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2b8dae-605b-4751-8eab-4de5487ba784 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:30.045765 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2b8dae-605b-4751-8eab-4de5487ba784 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:30.052499 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757cfca5-ecf0-44da-b740-8d5c3de03455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:30.053379 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757cfca5-ecf0-44da-b740-8d5c3de03455 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:30.053755 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757cfca5-ecf0-44da-b740-8d5c3de03455 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:30.054111 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757cfca5-ecf0-44da-b740-8d5c3de03455 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:30.064146 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757cfca5-ecf0-44da-b740-8d5c3de03455 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:30.065948 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757cfca5-ecf0-44da-b740-8d5c3de03455 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:30.072830 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 473/930] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Jun 24 08:28:29 2022] POST /identity/v3/endpoints => generated 347 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:28:30.097079 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-757cfca5-ecf0-44da-b740-8d5c3de03455 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 08:28:30.097079 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:30.097079 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:30.097079 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:30.097079 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:30.097079 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:30.097079 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:30.097079 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:30.097079 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:30.097079 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:30.097079 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:30.097079 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:30.097079 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:28:30.097079 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:28:30.097079 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:28:30.097079 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:28:30.098166 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:30.098166 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:30.098166 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:28:30.098166 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:28:30.098166 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:28:30.098166 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:28:30.098166 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:30.098166 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:30.098166 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:30.098166 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:30.098166 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:30.098166 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:30.098166 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:30.098166 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:30.098166 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:30.098166 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:30.098166 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:30.098166 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:30.099116 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:30.099116 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:30.099116 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:28:30.099116 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:28:30.099116 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:28:30.099116 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:28:30.099116 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:28:30.099116 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:28:30.099116 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 08:28:30.099116 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:30.099116 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 458/931] 10.222.0.13 () {56 vars in 1117 bytes} [Fri Jun 24 08:28:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:30.106259 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2b8dae-605b-4751-8eab-4de5487ba784 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:30.189702 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:30.238064 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2b8dae-605b-4751-8eab-4de5487ba784 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:30.275914 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98214b3-9f6b-4842-84b6-9bcfda62e017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:30.276953 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98214b3-9f6b-4842-84b6-9bcfda62e017 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:30.277464 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98214b3-9f6b-4842-84b6-9bcfda62e017 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:30.277965 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98214b3-9f6b-4842-84b6-9bcfda62e017 None admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:30.279017 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98214b3-9f6b-4842-84b6-9bcfda62e017 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:30.279523 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98214b3-9f6b-4842-84b6-9bcfda62e017 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:30.281141 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98214b3-9f6b-4842-84b6-9bcfda62e017 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:30.300104 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 474/932] 10.222.0.13 () {56 vars in 1149 bytes} [Fri Jun 24 08:28:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:30.308236 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757cfca5-ecf0-44da-b740-8d5c3de03455 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:30.428005 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757cfca5-ecf0-44da-b740-8d5c3de03455 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:30.462453 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11103d4a-c5d6-421b-aa8c-df866a0e5639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:30.463240 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11103d4a-c5d6-421b-aa8c-df866a0e5639 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:30.463520 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11103d4a-c5d6-421b-aa8c-df866a0e5639 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:30.463790 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11103d4a-c5d6-421b-aa8c-df866a0e5639 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:30.474591 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11103d4a-c5d6-421b-aa8c-df866a0e5639 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:30.476038 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11103d4a-c5d6-421b-aa8c-df866a0e5639 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:30.495018 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-11103d4a-c5d6-421b-aa8c-df866a0e5639 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:30.495018 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:30.495018 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:30.495018 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:30.495018 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:30.495018 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:30.495018 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:30.495018 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:30.495018 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:30.495018 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:30.495018 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:30.495018 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:30.495018 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:28:30.495018 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:28:30.495018 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:28:30.496040 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:28:30.496040 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:30.496040 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:30.496040 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:30.496040 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:30.496040 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:30.496040 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:30.496040 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:30.496040 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:30.496040 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:30.496040 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:30.496040 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:30.496040 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:30.496040 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:30.496040 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:30.496040 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:30.496040 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:30.496912 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:28:30.496912 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:28:30.496912 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:30.496912 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:30.496912 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 459/933] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:30.502205 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98214b3-9f6b-4842-84b6-9bcfda62e017 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:30.611477 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98214b3-9f6b-4842-84b6-9bcfda62e017 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:30.636350 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:30.651927 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58a979-e115-4ca2-a223-b8393125d09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:30.652981 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58a979-e115-4ca2-a223-b8393125d09c None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:30.653488 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58a979-e115-4ca2-a223-b8393125d09c None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:30.653995 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58a979-e115-4ca2-a223-b8393125d09c None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:30.654922 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f58a979-e115-4ca2-a223-b8393125d09c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:30.655522 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f58a979-e115-4ca2-a223-b8393125d09c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:30.657110 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f58a979-e115-4ca2-a223-b8393125d09c None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:30.670432 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 475/934] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:28:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:30.675651 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11103d4a-c5d6-421b-aa8c-df866a0e5639 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:30.701649 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:30.787025 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11103d4a-c5d6-421b-aa8c-df866a0e5639 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:30.821719 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69203d22-834a-4145-859c-de69954ec706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:30.822526 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69203d22-834a-4145-859c-de69954ec706 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:30.822806 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69203d22-834a-4145-859c-de69954ec706 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:30.823077 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69203d22-834a-4145-859c-de69954ec706 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:30.830206 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69203d22-834a-4145-859c-de69954ec706 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:30.831646 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69203d22-834a-4145-859c-de69954ec706 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:30.847065 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-69203d22-834a-4145-859c-de69954ec706 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:28:30.847065 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:30.847065 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:30.847065 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:30.847065 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:30.847065 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:30.847065 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:30.847065 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:30.847065 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:30.847065 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:30.847065 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:30.847065 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:30.847065 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:28:30.847065 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:28:30.847065 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:28:30.848016 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:28:30.848016 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:30.848016 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:30.848016 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:30.848016 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:30.848016 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:30.848016 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:30.848016 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:30.848016 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:30.848016 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:30.848016 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:30.848016 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:30.848016 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:30.848016 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:30.848016 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:30.848016 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:30.848016 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:30.848908 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:28:30.848908 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:28:30.848908 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:28:30.848908 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:28:30.848908 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:28:30.848908 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:28:30.848908 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:28:30.848908 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:30.848908 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 460/935] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:28:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:30.854353 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58a979-e115-4ca2-a223-b8393125d09c None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:30.962990 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58a979-e115-4ca2-a223-b8393125d09c None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:31.003851 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0752b6b5-6b63-499d-8275-57fc8930e3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:31.004519 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752b6b5-6b63-499d-8275-57fc8930e3a9 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:31.004799 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752b6b5-6b63-499d-8275-57fc8930e3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:31.005075 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752b6b5-6b63-499d-8275-57fc8930e3a9 None admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:31.005907 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0752b6b5-6b63-499d-8275-57fc8930e3a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:31.006217 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0752b6b5-6b63-499d-8275-57fc8930e3a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:31.007674 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0752b6b5-6b63-499d-8275-57fc8930e3a9 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:31.021543 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 476/936] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Jun 24 08:28:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:31.028750 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69203d22-834a-4145-859c-de69954ec706 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:31.039171 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e9c18c-ef87-479e-8453-fe9cea499723 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:31.039466 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e9c18c-ef87-479e-8453-fe9cea499723 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:31.039951 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e9c18c-ef87-479e-8453-fe9cea499723 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:31.045243 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 477/937] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:31.046867 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3654a48-185f-47f1-aea3-83fe4c49e9c8 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:31.047156 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3654a48-185f-47f1-aea3-83fe4c49e9c8 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:31.047433 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3654a48-185f-47f1-aea3-83fe4c49e9c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:31.065216 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-a3654a48-185f-47f1-aea3-83fe4c49e9c8 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:31.066509 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-a3654a48-185f-47f1-aea3-83fe4c49e9c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:31.102974 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:31.118480 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:31.144783 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69203d22-834a-4145-859c-de69954ec706 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:31.146956 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 478/938] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 08:28:31.168888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc7369c-8c75-4d2a-acc6-c6c413edf289 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:31.169190 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc7369c-8c75-4d2a-acc6-c6c413edf289 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:31.169465 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc7369c-8c75-4d2a-acc6-c6c413edf289 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:31.170988 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 479/939] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:28:31.171543 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc7369c-8c75-4d2a-acc6-c6c413edf289 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:31.181477 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74eb2d8-0b12-4724-90e5-17b7636db8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:31.182530 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74eb2d8-0b12-4724-90e5-17b7636db8c8 None admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:31.182818 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74eb2d8-0b12-4724-90e5-17b7636db8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:31.183219 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74eb2d8-0b12-4724-90e5-17b7636db8c8 None admin] PATH_INFO: `/v3/projects/180b0af5863349e99ae412b07c1e9465/users/85d9c4c57681447c898ebfd7990b05b6/roles/f9658866044e4fdeaddcc91781166faf` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:31.185573 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74eb2d8-0b12-4724-90e5-17b7636db8c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=180b0af5863349e99ae412b07c1e9465, user_id=85d9c4c57681447c898ebfd7990b05b6, role_id=f9658866044e4fdeaddcc91781166faf)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:31.187211 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74eb2d8-0b12-4724-90e5-17b7636db8c8 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:31.208804 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 461/940] 10.222.0.13 () {58 vars in 1439 bytes} [Fri Jun 24 08:28:31 2022] PUT /identity/v3/projects/180b0af5863349e99ae412b07c1e9465/users/85d9c4c57681447c898ebfd7990b05b6/roles/f9658866044e4fdeaddcc91781166faf => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:28:31.208804 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bfcbb5-adb9-4e96-94bc-a340bba0ed32 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:31.208804 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bfcbb5-adb9-4e96-94bc-a340bba0ed32 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:31.208804 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bfcbb5-adb9-4e96-94bc-a340bba0ed32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:31.223499 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-99bfcbb5-adb9-4e96-94bc-a340bba0ed32 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:31.224780 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-99bfcbb5-adb9-4e96-94bc-a340bba0ed32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:31.304410 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc7369c-8c75-4d2a-acc6-c6c413edf289 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:31.310439 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 462/941] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:28:31.332881 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bfcbb5-adb9-4e96-94bc-a340bba0ed32 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:31.346443 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb3b00-326c-4201-aab0-c3c3e69b4d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:31.347274 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb3b00-326c-4201-aab0-c3c3e69b4d44 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:31.347562 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb3b00-326c-4201-aab0-c3c3e69b4d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:31.347835 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb3b00-326c-4201-aab0-c3c3e69b4d44 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:31.348501 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abb3b00-326c-4201-aab0-c3c3e69b4d44 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:31.349943 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abb3b00-326c-4201-aab0-c3c3e69b4d44 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:31.363235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-4abb3b00-326c-4201-aab0-c3c3e69b4d44 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 08:28:31.363235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:31.363235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 08:28:31.363235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 08:28:31.363235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 08:28:31.363235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 08:28:31.363235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 08:28:31.363235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:28:31.363235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 08:28:31.363235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:31.363235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:28:31.363235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:31.363235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:31.363235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:31.363235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:31.363235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:31.363235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:31.364301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:31.364301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:31.364301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:31.364301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:31.364301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:31.364301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:31.364301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 08:28:31.364301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 08:28:31.364301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 08:28:31.364301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 08:28:31.364301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:31.364301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:31.364301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:31.364301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:31.364301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:31.364301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:31.365164 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:31.365164 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:31.365164 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:31.365164 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:31.365164 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:31.365164 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:31.365164 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:31.365164 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:31.365164 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:31.365164 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:31.365164 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 08:28:31.365164 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:28:31.365164 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 08:28:31.365164 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:31.370109 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 480/942] 10.222.0.13 () {56 vars in 1126 bytes} [Fri Jun 24 08:28:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:31.371690 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb3b00-326c-4201-aab0-c3c3e69b4d44 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:31.419741 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:31.432708 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:31.466193 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bfcbb5-adb9-4e96-94bc-a340bba0ed32 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:31.501771 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb3b00-326c-4201-aab0-c3c3e69b4d44 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:31.515182 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677be8bc-0d94-40eb-97ef-1f9122da8eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:31.515874 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677be8bc-0d94-40eb-97ef-1f9122da8eec None admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:31.516179 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677be8bc-0d94-40eb-97ef-1f9122da8eec None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:31.516461 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677be8bc-0d94-40eb-97ef-1f9122da8eec None admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:31.517229 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677be8bc-0d94-40eb-97ef-1f9122da8eec None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:31.518795 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677be8bc-0d94-40eb-97ef-1f9122da8eec None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:31.520208 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: WARNING keystone.server.flask.common [None req-677be8bc-0d94-40eb-97ef-1f9122da8eec None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 24 08:28:31.540192 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.sql.core [None req-677be8bc-0d94-40eb-97ef-1f9122da8eec None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 24 08:28:31.540192 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 24 08:28:31.540192 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [parameters: {'id': 'd7bbbb214287462f8ac56b398845aed2', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 24 08:28:31.540192 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80425) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 24 08:28:31.550507 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d478b-95d7-4467-b02a-9899c10b134b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:31.551522 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d478b-95d7-4467-b02a-9899c10b134b None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:31.552068 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d478b-95d7-4467-b02a-9899c10b134b None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:31.552545 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d478b-95d7-4467-b02a-9899c10b134b None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:31.553465 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094d478b-95d7-4467-b02a-9899c10b134b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:31.553996 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094d478b-95d7-4467-b02a-9899c10b134b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:31.555734 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094d478b-95d7-4467-b02a-9899c10b134b None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:31.558369 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-677be8bc-0d94-40eb-97ef-1f9122da8eec None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d7bbbb214287462f8ac56b398845aed2.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d7bbbb214287462f8ac56b398845aed2. Jun 24 08:28:31.558369 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:31.558369 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 08:28:31.558369 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 08:28:31.558369 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 08:28:31.558369 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 08:28:31.558369 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 08:28:31.558369 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 08:28:31.558369 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 08:28:31.558369 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application conn.query(q) Jun 24 08:28:31.558369 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 08:28:31.558369 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 08:28:31.558369 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 08:28:31.559219 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application result.read() Jun 24 08:28:31.559219 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 08:28:31.559219 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 08:28:31.559219 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 08:28:31.559219 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 08:28:31.559219 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 08:28:31.559219 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 08:28:31.559219 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 08:28:31.559219 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 08:28:31.559219 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 24 08:28:31.559219 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:31.559219 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 24 08:28:31.559219 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:31.559219 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:31.559219 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 24 08:28:31.559219 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 24 08:28:31.559219 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 24 08:28:31.559219 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 24 08:28:31.560184 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 08:28:31.560184 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application next(self.gen) Jun 24 08:28:31.560184 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 24 08:28:31.560184 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application yield resource Jun 24 08:28:31.560184 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 08:28:31.560184 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application next(self.gen) Jun 24 08:28:31.560184 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 24 08:28:31.560184 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self.session.rollback() Jun 24 08:28:31.560184 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 24 08:28:31.560184 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self.force_reraise() Jun 24 08:28:31.560184 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 24 08:28:31.560184 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise self.value Jun 24 08:28:31.560184 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 24 08:28:31.560184 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 24 08:28:31.560184 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 24 08:28:31.560184 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application session.commit() Jun 24 08:28:31.560184 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 24 08:28:31.561072 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 24 08:28:31.561072 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 24 08:28:31.561072 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._prepare_impl() Jun 24 08:28:31.561072 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 24 08:28:31.561072 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self.session.flush() Jun 24 08:28:31.561072 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 24 08:28:31.561072 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._flush(objects) Jun 24 08:28:31.561072 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 24 08:28:31.561072 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 24 08:28:31.561072 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 24 08:28:31.561072 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application compat.raise_( Jun 24 08:28:31.561072 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 08:28:31.561072 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception Jun 24 08:28:31.561072 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 24 08:28:31.561072 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application flush_context.execute() Jun 24 08:28:31.561072 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 24 08:28:31.561072 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rec.execute(self) Jun 24 08:28:31.562210 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 24 08:28:31.562210 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 24 08:28:31.562210 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 24 08:28:31.562210 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 24 08:28:31.562210 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 24 08:28:31.562210 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 24 08:28:31.562210 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 24 08:28:31.562210 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 24 08:28:31.562210 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 24 08:28:31.562210 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 24 08:28:31.562210 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 24 08:28:31.562210 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 24 08:28:31.562210 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 24 08:28:31.562210 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 24 08:28:31.562210 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 24 08:28:31.563142 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 24 08:28:31.563142 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 08:28:31.563142 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception Jun 24 08:28:31.563142 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 08:28:31.563142 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 08:28:31.563142 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 08:28:31.563142 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 08:28:31.563142 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 08:28:31.563142 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 08:28:31.563142 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 08:28:31.563142 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application conn.query(q) Jun 24 08:28:31.563142 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 08:28:31.563142 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 08:28:31.563142 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 08:28:31.563142 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application result.read() Jun 24 08:28:31.563142 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 08:28:31.563142 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 08:28:31.564216 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 08:28:31.564216 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 08:28:31.564216 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 08:28:31.564216 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 08:28:31.564216 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 08:28:31.564216 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 08:28:31.564216 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 24 08:28:31.564216 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 24 08:28:31.564216 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [parameters: {'id': 'd7bbbb214287462f8ac56b398845aed2', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 24 08:28:31.564216 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 24 08:28:31.564216 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:31.564216 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:28:31.564216 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:31.564216 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:31.564216 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 24 08:28:31.565921 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 24 08:28:31.565921 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 24 08:28:31.565921 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 24 08:28:31.565921 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 24 08:28:31.565921 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:31.565921 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:28:31.565921 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:31.565921 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:31.565921 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:31.565921 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:31.565921 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:31.565921 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:31.565921 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:31.565921 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:31.565921 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:31.565921 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:31.565921 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:31.568149 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:31.568149 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 24 08:28:31.568149 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 24 08:28:31.568149 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:31.568149 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:31.568149 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 24 08:28:31.568149 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 24 08:28:31.568149 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d7bbbb214287462f8ac56b398845aed2. Jun 24 08:28:31.568149 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:31.568149 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 463/943] 10.222.0.13 () {60 vars in 1140 bytes} [Fri Jun 24 08:28:31 2022] POST /identity/v3/projects => generated 262 bytes in 228 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:28:31.568149 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677be8bc-0d94-40eb-97ef-1f9122da8eec None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:31.576282 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 481/944] 10.222.0.13 () {56 vars in 1122 bytes} [Fri Jun 24 08:28:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:31.582845 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d478b-95d7-4467-b02a-9899c10b134b None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:31.701566 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677be8bc-0d94-40eb-97ef-1f9122da8eec None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:31.702416 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:31.706510 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d478b-95d7-4467-b02a-9899c10b134b None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:31.735915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:31.743974 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4be7d1-02e5-47de-98de-40b1ed373283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:31.744732 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4be7d1-02e5-47de-98de-40b1ed373283 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:31.745052 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4be7d1-02e5-47de-98de-40b1ed373283 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:31.745275 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4be7d1-02e5-47de-98de-40b1ed373283 None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:31.746029 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4be7d1-02e5-47de-98de-40b1ed373283 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:31.746331 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4be7d1-02e5-47de-98de-40b1ed373283 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:31.747698 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4be7d1-02e5-47de-98de-40b1ed373283 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:31.748455 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a26a4c-9849-4442-a2e9-ea6f9d93283d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:31.749515 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a26a4c-9849-4442-a2e9-ea6f9d93283d None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:31.750035 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a26a4c-9849-4442-a2e9-ea6f9d93283d None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:31.750503 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a26a4c-9849-4442-a2e9-ea6f9d93283d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:31.755029 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 482/945] 10.222.0.13 () {56 vars in 1122 bytes} [Fri Jun 24 08:28:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:31.762766 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a26a4c-9849-4442-a2e9-ea6f9d93283d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:31.763679 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a26a4c-9849-4442-a2e9-ea6f9d93283d None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:31.764951 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4be7d1-02e5-47de-98de-40b1ed373283 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:31.785090 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-e8a26a4c-9849-4442-a2e9-ea6f9d93283d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:28:31.785090 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:31.785090 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:31.785090 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:31.785090 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:31.785090 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:31.785090 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:31.785090 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:31.785090 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:31.785090 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:31.785090 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:31.785090 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:31.785090 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:28:31.785090 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:28:31.785090 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:28:31.786070 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:28:31.786070 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:31.786070 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:31.786070 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:31.786070 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:31.786070 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:31.786070 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:31.786070 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:31.786070 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:31.786070 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:31.786070 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:31.786070 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:31.786070 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:31.786070 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:31.786070 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:31.786070 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:31.786070 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:31.786952 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:28:31.786952 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:28:31.786952 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:28:31.786952 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:28:31.786952 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:28:31.786952 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:28:31.786952 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:28:31.786952 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:31.787404 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 464/946] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:28:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:28:31.794658 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a26a4c-9849-4442-a2e9-ea6f9d93283d None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:31.876709 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4be7d1-02e5-47de-98de-40b1ed373283 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:31.918077 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0a291-35fb-4e1e-84e6-1c3763e1d6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:31.919077 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0a291-35fb-4e1e-84e6-1c3763e1d6f8 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:31.919364 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0a291-35fb-4e1e-84e6-1c3763e1d6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:31.919641 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0a291-35fb-4e1e-84e6-1c3763e1d6f8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:31.920744 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db0a291-35fb-4e1e-84e6-1c3763e1d6f8 None admin] RBAC: Adding query filter params (interface=public, service_id=ce4106318b154293be53a8b74d6bf50f, region_id=RegionOne) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:31.921061 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db0a291-35fb-4e1e-84e6-1c3763e1d6f8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:31.922455 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db0a291-35fb-4e1e-84e6-1c3763e1d6f8 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:31.922856 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a26a4c-9849-4442-a2e9-ea6f9d93283d None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:31.929731 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 483/947] 10.222.0.13 () {56 vars in 1251 bytes} [Fri Jun 24 08:28:31 2022] GET /identity/v3/endpoints?service_id=ce4106318b154293be53a8b74d6bf50f&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:31.937443 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0a291-35fb-4e1e-84e6-1c3763e1d6f8 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:31.973239 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:31.979833 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d1eabc-86ca-4279-9913-7e2c54322e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:31.981042 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d1eabc-86ca-4279-9913-7e2c54322e75 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:31.981543 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d1eabc-86ca-4279-9913-7e2c54322e75 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:31.982096 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d1eabc-86ca-4279-9913-7e2c54322e75 None admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:31.983353 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d1eabc-86ca-4279-9913-7e2c54322e75 None admin] RBAC: Adding query filter params (name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:31.984793 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d1eabc-86ca-4279-9913-7e2c54322e75 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:31.986478 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d1eabc-86ca-4279-9913-7e2c54322e75 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:32.026069 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 465/948] 10.222.0.13 () {56 vars in 1112 bytes} [Fri Jun 24 08:28:31 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:32.069642 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0a291-35fb-4e1e-84e6-1c3763e1d6f8 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:32.107882 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf5f062-d8b0-4b44-b186-4151527984d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:32.108564 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5f062-d8b0-4b44-b186-4151527984d7 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:32.108849 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5f062-d8b0-4b44-b186-4151527984d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:32.109127 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5f062-d8b0-4b44-b186-4151527984d7 None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:32.109850 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf5f062-d8b0-4b44-b186-4151527984d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:32.111312 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf5f062-d8b0-4b44-b186-4151527984d7 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:32.119278 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 484/949] 10.222.0.13 () {56 vars in 1087 bytes} [Fri Jun 24 08:28:31 2022] GET /identity/v3/services => generated 1497 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:28:32.409627 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37424f15-8657-417d-a48d-5f2a50b58dc0 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:32.410199 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37424f15-8657-417d-a48d-5f2a50b58dc0 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:32.410199 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37424f15-8657-417d-a48d-5f2a50b58dc0 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:32.411952 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 466/950] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:32.417298 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe6401-5165-4aa2-9b89-9bc3d248145c None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:32.417746 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe6401-5165-4aa2-9b89-9bc3d248145c None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:32.417946 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe6401-5165-4aa2-9b89-9bc3d248145c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:32.435915 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-6bbe6401-5165-4aa2-9b89-9bc3d248145c None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:32.437134 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-6bbe6401-5165-4aa2-9b89-9bc3d248145c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:32.510409 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 485/951] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:28:32.532526 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37424f15-8657-417d-a48d-5f2a50b58dc0 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:32.589936 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:32.646184 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37424f15-8657-417d-a48d-5f2a50b58dc0 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:32.682309 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe238477-1141-4bfe-9324-fc7768931a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:32.683455 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe238477-1141-4bfe-9324-fc7768931a52 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:32.683818 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe238477-1141-4bfe-9324-fc7768931a52 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:32.684185 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe238477-1141-4bfe-9324-fc7768931a52 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:32.685362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe238477-1141-4bfe-9324-fc7768931a52 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:32.687444 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe238477-1141-4bfe-9324-fc7768931a52 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:32.707350 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-fe238477-1141-4bfe-9324-fc7768931a52 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 08:28:32.707350 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:32.707350 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 08:28:32.707350 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 08:28:32.707350 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 08:28:32.707350 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 08:28:32.707350 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 08:28:32.707350 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:28:32.707350 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 08:28:32.707350 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:32.707350 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:28:32.707350 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:32.707350 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:32.707350 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:32.707350 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:32.707350 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:32.707350 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:32.708427 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:32.708427 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:32.708427 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:32.708427 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:32.708427 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:32.708427 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:32.708427 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 08:28:32.708427 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 08:28:32.708427 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 08:28:32.708427 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 08:28:32.708427 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:32.708427 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:32.708427 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:32.708427 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:32.708427 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:32.708427 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:32.709334 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:32.709334 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:32.709334 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:32.709334 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:32.709334 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:32.709334 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:32.709334 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:32.709334 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:32.709334 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:32.709334 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:32.709334 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 08:28:32.709334 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:28:32.709334 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 08:28:32.709334 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:32.709334 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 467/952] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:28:32 2022] GET /identity/v3/services/compute => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:32.715223 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbe6401-5165-4aa2-9b89-9bc3d248145c None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:32.737398 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:32.819590 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbe6401-5165-4aa2-9b89-9bc3d248145c None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:32.858979 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351b2527-3524-4bd6-aa1d-0c5bc38e9f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:32.859717 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b2527-3524-4bd6-aa1d-0c5bc38e9f39 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:32.859998 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b2527-3524-4bd6-aa1d-0c5bc38e9f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:32.860883 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b2527-3524-4bd6-aa1d-0c5bc38e9f39 None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:32.861602 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351b2527-3524-4bd6-aa1d-0c5bc38e9f39 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:32.862009 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351b2527-3524-4bd6-aa1d-0c5bc38e9f39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:32.864006 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351b2527-3524-4bd6-aa1d-0c5bc38e9f39 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:32.870195 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 486/953] 10.222.0.13 () {56 vars in 1112 bytes} [Fri Jun 24 08:28:32 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:32.875793 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe238477-1141-4bfe-9324-fc7768931a52 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:32.989386 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe238477-1141-4bfe-9324-fc7768931a52 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:33.002838 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:33.025613 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e28797-e680-46a7-9f91-76012607b7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:33.026302 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e28797-e680-46a7-9f91-76012607b7ed None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:33.026584 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e28797-e680-46a7-9f91-76012607b7ed None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:33.026857 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e28797-e680-46a7-9f91-76012607b7ed None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:33.027557 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e28797-e680-46a7-9f91-76012607b7ed None admin] RBAC: Adding query filter params (type=compute) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:33.027856 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e28797-e680-46a7-9f91-76012607b7ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:33.029228 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e28797-e680-46a7-9f91-76012607b7ed None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:33.034319 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 468/954] 10.222.0.13 () {56 vars in 1112 bytes} [Fri Jun 24 08:28:32 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:33.392324 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43db573-83d7-44bb-9951-80415ca8ceb2 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:33.392666 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43db573-83d7-44bb-9951-80415ca8ceb2 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:33.392881 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43db573-83d7-44bb-9951-80415ca8ceb2 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:33.394568 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 487/955] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:33.400378 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34936f-be8c-4118-b96a-928b8fd4a95b None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:33.400669 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34936f-be8c-4118-b96a-928b8fd4a95b None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:33.400944 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34936f-be8c-4118-b96a-928b8fd4a95b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:33.420224 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-7f34936f-be8c-4118-b96a-928b8fd4a95b None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:33.421732 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-7f34936f-be8c-4118-b96a-928b8fd4a95b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:33.492117 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 469/956] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:28:33.515398 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43db573-83d7-44bb-9951-80415ca8ceb2 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:33.604484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:33.623086 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43db573-83d7-44bb-9951-80415ca8ceb2 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:33.657984 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f01c2-a7ce-490d-ae62-298ebfe8ec24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:33.658841 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f01c2-a7ce-490d-ae62-298ebfe8ec24 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:33.659122 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f01c2-a7ce-490d-ae62-298ebfe8ec24 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:33.659380 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f01c2-a7ce-490d-ae62-298ebfe8ec24 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:33.672950 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6f01c2-a7ce-490d-ae62-298ebfe8ec24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:33.681725 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-2e6f01c2-a7ce-490d-ae62-298ebfe8ec24 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 08:28:33.681725 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:33.681725 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:33.681725 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:33.681725 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:33.681725 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:33.681725 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:33.681725 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:33.681725 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:33.681725 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:33.681725 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:33.681725 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:33.681725 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:33.681725 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:33.681725 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:33.681725 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:33.682735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:33.682735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:33.682735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:33.682735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:33.682735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:33.682735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:33.682735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:33.682735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:33.682735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:33.682735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:33.682735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:33.682735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:33.682735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:33.682735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:33.682735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:33.682735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:33.683601 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:33.683601 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:33.683601 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:33.683601 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:33.683601 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:33.683601 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:33.683601 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:33.683601 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:33.683601 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:33.683601 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:33.683601 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 08:28:33.683601 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:33.683601 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 488/957] 10.222.0.13 () {56 vars in 1120 bytes} [Fri Jun 24 08:28:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:33.689206 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f34936f-be8c-4118-b96a-928b8fd4a95b None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:33.799662 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f34936f-be8c-4118-b96a-928b8fd4a95b None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:33.807762 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:33.836264 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e498944-96e9-44f9-abc5-fa3c6f745089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:33.837370 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e498944-96e9-44f9-abc5-fa3c6f745089 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:33.837714 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e498944-96e9-44f9-abc5-fa3c6f745089 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:33.838005 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e498944-96e9-44f9-abc5-fa3c6f745089 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:33.838748 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e498944-96e9-44f9-abc5-fa3c6f745089 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:33.839052 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e498944-96e9-44f9-abc5-fa3c6f745089 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:33.840427 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e498944-96e9-44f9-abc5-fa3c6f745089 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:33.849432 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 470/958] 10.222.0.13 () {56 vars in 1115 bytes} [Fri Jun 24 08:28:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:33.855239 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f01c2-a7ce-490d-ae62-298ebfe8ec24 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:33.963280 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f01c2-a7ce-490d-ae62-298ebfe8ec24 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:33.999879 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02dd5ff-f176-4459-88cc-310502b36ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:34.000729 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02dd5ff-f176-4459-88cc-310502b36ff9 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:34.000995 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02dd5ff-f176-4459-88cc-310502b36ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:34.001321 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02dd5ff-f176-4459-88cc-310502b36ff9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:34.012344 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02dd5ff-f176-4459-88cc-310502b36ff9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:34.013841 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02dd5ff-f176-4459-88cc-310502b36ff9 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:34.020342 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:34.031942 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-c02dd5ff-f176-4459-88cc-310502b36ff9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:34.031942 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:34.031942 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:34.031942 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:34.031942 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:34.031942 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:34.031942 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:34.031942 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:34.031942 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:34.031942 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:34.031942 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:34.031942 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:34.031942 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:28:34.031942 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:28:34.031942 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:28:34.034038 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:28:34.034038 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:34.034038 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:34.034038 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:34.034038 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:34.034038 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:34.034038 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:34.034038 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:34.034038 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:34.034038 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:34.034038 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:34.034038 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:34.034038 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:34.034038 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:34.034038 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:34.034038 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:34.034038 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:34.036130 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:28:34.036130 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:28:34.036130 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:34.036130 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:34.036130 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 489/959] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:34.042762 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e498944-96e9-44f9-abc5-fa3c6f745089 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:34.157593 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e498944-96e9-44f9-abc5-fa3c6f745089 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:34.200501 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65315664-5d2d-48c3-a1ee-0ecf88028cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:34.201231 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65315664-5d2d-48c3-a1ee-0ecf88028cfa None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:34.201520 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65315664-5d2d-48c3-a1ee-0ecf88028cfa None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:34.201829 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65315664-5d2d-48c3-a1ee-0ecf88028cfa None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:34.202567 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65315664-5d2d-48c3-a1ee-0ecf88028cfa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:34.202881 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65315664-5d2d-48c3-a1ee-0ecf88028cfa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:34.204355 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65315664-5d2d-48c3-a1ee-0ecf88028cfa None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:34.218685 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 471/960] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:28:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:34.223756 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02dd5ff-f176-4459-88cc-310502b36ff9 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:34.320535 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0527aa3-9a1b-413b-a672-fda8b357870c None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:34.321513 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0527aa3-9a1b-413b-a672-fda8b357870c None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:34.322018 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0527aa3-9a1b-413b-a672-fda8b357870c None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:34.323549 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 472/961] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:34.324550 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd60005-28b0-4a43-9526-163c71415856 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:34.324837 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd60005-28b0-4a43-9526-163c71415856 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:34.325112 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd60005-28b0-4a43-9526-163c71415856 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:34.326613 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 473/962] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:34.334638 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210c6aa2-51f2-47c0-8494-f2e1e9adff6d None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:34.335061 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210c6aa2-51f2-47c0-8494-f2e1e9adff6d None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:34.335221 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210c6aa2-51f2-47c0-8494-f2e1e9adff6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:34.339068 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02dd5ff-f176-4459-88cc-310502b36ff9 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:34.352206 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-210c6aa2-51f2-47c0-8494-f2e1e9adff6d None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:34.357209 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-210c6aa2-51f2-47c0-8494-f2e1e9adff6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:34.379469 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:34.386768 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8ebeca-5440-4eaa-b6a2-1f0d2fca4205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:34.387527 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8ebeca-5440-4eaa-b6a2-1f0d2fca4205 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:34.387801 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8ebeca-5440-4eaa-b6a2-1f0d2fca4205 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:34.388078 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8ebeca-5440-4eaa-b6a2-1f0d2fca4205 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:34.396196 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8ebeca-5440-4eaa-b6a2-1f0d2fca4205 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:34.397620 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8ebeca-5440-4eaa-b6a2-1f0d2fca4205 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:34.412159 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-ae8ebeca-5440-4eaa-b6a2-1f0d2fca4205 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 08:28:34.412159 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:34.412159 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:34.412159 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:34.412159 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:34.412159 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:34.412159 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:34.412159 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:34.412159 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:34.412159 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:34.412159 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:34.412159 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:34.412159 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:28:34.412159 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:28:34.412159 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:28:34.412159 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:28:34.413074 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:34.413074 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:34.413074 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:28:34.413074 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:28:34.413074 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:28:34.413074 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:28:34.413074 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:34.413074 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:34.413074 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:34.413074 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:34.413074 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:34.413074 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:34.413074 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:34.413074 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:34.413074 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:34.413074 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:34.413074 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:34.413074 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:34.415091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:34.415091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:34.415091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:28:34.415091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:28:34.415091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:28:34.415091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:28:34.415091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:28:34.415091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:28:34.415091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 08:28:34.415091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:34.415091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 490/963] 10.222.0.13 () {56 vars in 1117 bytes} [Fri Jun 24 08:28:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:34.415091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5516d-6498-4ba8-b753-ed2dd3f94308 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:34.415091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5516d-6498-4ba8-b753-ed2dd3f94308 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:34.416043 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5516d-6498-4ba8-b753-ed2dd3f94308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:34.433006 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-1ef5516d-6498-4ba8-b753-ed2dd3f94308 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:34.433523 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 474/964] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:34 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:28:34.434226 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-1ef5516d-6498-4ba8-b753-ed2dd3f94308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:34.435070 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210c6aa2-51f2-47c0-8494-f2e1e9adff6d None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:34.490928 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:34.510527 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 491/965] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:34 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:28:34.511196 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef5516d-6498-4ba8-b753-ed2dd3f94308 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:34.565995 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210c6aa2-51f2-47c0-8494-f2e1e9adff6d None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:34.605479 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ef0797-acb5-403f-ab8b-3ef452d451ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:34.606181 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ef0797-acb5-403f-ab8b-3ef452d451ea None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:34.606472 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ef0797-acb5-403f-ab8b-3ef452d451ea None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:34.606753 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ef0797-acb5-403f-ab8b-3ef452d451ea None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:34.607585 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ef0797-acb5-403f-ab8b-3ef452d451ea None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:34.608027 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ef0797-acb5-403f-ab8b-3ef452d451ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:34.609323 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ef0797-acb5-403f-ab8b-3ef452d451ea None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:34.616917 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef5516d-6498-4ba8-b753-ed2dd3f94308 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:34.632786 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 475/966] 10.222.0.13 () {56 vars in 1149 bytes} [Fri Jun 24 08:28:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:34.632786 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ef0797-acb5-403f-ab8b-3ef452d451ea None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:34.651120 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:34.655687 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e17d8d-63cb-4d9e-b82b-c887c80c7f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:34.656763 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e17d8d-63cb-4d9e-b82b-c887c80c7f46 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:34.657139 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e17d8d-63cb-4d9e-b82b-c887c80c7f46 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:34.657317 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e17d8d-63cb-4d9e-b82b-c887c80c7f46 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:34.658018 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e17d8d-63cb-4d9e-b82b-c887c80c7f46 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:34.659379 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e17d8d-63cb-4d9e-b82b-c887c80c7f46 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:34.671477 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-00e17d8d-63cb-4d9e-b82b-c887c80c7f46 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 08:28:34.671477 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:34.671477 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 08:28:34.671477 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 08:28:34.671477 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 08:28:34.671477 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 08:28:34.671477 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 08:28:34.671477 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:28:34.671477 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 08:28:34.671477 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:34.671477 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:28:34.671477 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:34.671477 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:34.671477 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:34.671477 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:34.671477 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:34.671477 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:34.672541 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:34.672541 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:34.672541 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:34.672541 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:34.672541 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:34.672541 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:34.672541 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 08:28:34.672541 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 08:28:34.672541 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 08:28:34.672541 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 08:28:34.672541 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:34.672541 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:34.672541 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:34.672541 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:34.672541 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:34.672541 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:34.673382 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:34.673382 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:34.673382 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:34.673382 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:34.673382 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:34.673382 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:34.673382 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:34.673382 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:34.673382 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:34.673382 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:34.673382 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 08:28:34.673382 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:28:34.673382 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 08:28:34.673382 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:34.673382 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 492/967] 10.222.0.13 () {56 vars in 1126 bytes} [Fri Jun 24 08:28:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:34.674672 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e17d8d-63cb-4d9e-b82b-c887c80c7f46 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:34.733252 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:34.748051 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ef0797-acb5-403f-ab8b-3ef452d451ea None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:34.784850 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466cb2da-a65a-408d-8c65-032e514e952a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:34.785521 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466cb2da-a65a-408d-8c65-032e514e952a None admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:34.785834 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466cb2da-a65a-408d-8c65-032e514e952a None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:34.786114 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466cb2da-a65a-408d-8c65-032e514e952a None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:34.786811 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466cb2da-a65a-408d-8c65-032e514e952a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:34.788248 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466cb2da-a65a-408d-8c65-032e514e952a None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:34.789343 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e17d8d-63cb-4d9e-b82b-c887c80c7f46 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:34.803187 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.sql.core [None req-466cb2da-a65a-408d-8c65-032e514e952a None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 24 08:28:34.803187 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 24 08:28:34.803187 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [parameters: {'user_id': 'e9d0acf950e84844b188b46b710814b6', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 24 08:28:34.803187 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80425) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 24 08:28:34.811373 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-466cb2da-a65a-408d-8c65-032e514e952a None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 24 08:28:34.811373 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:34.811373 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 08:28:34.811373 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 08:28:34.811373 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 08:28:34.811373 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 08:28:34.811373 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 08:28:34.811373 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 08:28:34.811373 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 08:28:34.811373 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application conn.query(q) Jun 24 08:28:34.811373 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 08:28:34.811373 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 08:28:34.811373 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 08:28:34.811373 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application result.read() Jun 24 08:28:34.811373 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 08:28:34.812276 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 08:28:34.812276 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 08:28:34.812276 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 08:28:34.812276 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 08:28:34.812276 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 08:28:34.812276 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 08:28:34.812276 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 08:28:34.812276 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 24 08:28:34.812276 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:34.812276 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 24 08:28:34.812276 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:34.812276 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:34.812276 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 24 08:28:34.812276 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 24 08:28:34.812276 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 24 08:28:34.812276 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 24 08:28:34.812276 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 08:28:34.812276 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application next(self.gen) Jun 24 08:28:34.813196 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 24 08:28:34.813196 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application yield resource Jun 24 08:28:34.813196 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 08:28:34.813196 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application next(self.gen) Jun 24 08:28:34.813196 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 24 08:28:34.813196 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self.session.rollback() Jun 24 08:28:34.813196 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 24 08:28:34.813196 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self.force_reraise() Jun 24 08:28:34.813196 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 24 08:28:34.813196 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise self.value Jun 24 08:28:34.813196 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 24 08:28:34.813196 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 24 08:28:34.813196 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 24 08:28:34.813196 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application session.commit() Jun 24 08:28:34.813196 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 24 08:28:34.813196 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 24 08:28:34.813196 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 24 08:28:34.814217 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._prepare_impl() Jun 24 08:28:34.814217 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 24 08:28:34.814217 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self.session.flush() Jun 24 08:28:34.814217 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 24 08:28:34.814217 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._flush(objects) Jun 24 08:28:34.814217 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 24 08:28:34.814217 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 24 08:28:34.814217 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 24 08:28:34.814217 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application compat.raise_( Jun 24 08:28:34.814217 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 08:28:34.814217 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception Jun 24 08:28:34.814217 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 24 08:28:34.814217 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application flush_context.execute() Jun 24 08:28:34.814217 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 24 08:28:34.814217 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rec.execute(self) Jun 24 08:28:34.814217 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 24 08:28:34.814217 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 24 08:28:34.815118 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 24 08:28:34.815118 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 24 08:28:34.815118 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 24 08:28:34.815118 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 24 08:28:34.815118 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 24 08:28:34.815118 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 24 08:28:34.815118 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 24 08:28:34.815118 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 24 08:28:34.815118 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 24 08:28:34.815118 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 24 08:28:34.815118 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 24 08:28:34.815118 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 24 08:28:34.815118 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 24 08:28:34.815118 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 24 08:28:34.815118 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 08:28:34.815118 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception Jun 24 08:28:34.815961 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 08:28:34.815961 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 08:28:34.815961 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 08:28:34.815961 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 08:28:34.815961 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 08:28:34.815961 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 08:28:34.815961 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 08:28:34.815961 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application conn.query(q) Jun 24 08:28:34.815961 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 08:28:34.815961 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 08:28:34.815961 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 08:28:34.815961 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application result.read() Jun 24 08:28:34.815961 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 08:28:34.815961 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 08:28:34.815961 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 08:28:34.815961 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 08:28:34.816811 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 08:28:34.816811 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 08:28:34.816811 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 08:28:34.816811 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 08:28:34.816811 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 24 08:28:34.816811 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 24 08:28:34.816811 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [parameters: {'user_id': 'e9d0acf950e84844b188b46b710814b6', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 24 08:28:34.816811 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 24 08:28:34.816811 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:34.816811 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:28:34.816811 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:34.816811 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:34.816811 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:34.816811 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:34.816811 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:34.819486 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:34.819486 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:34.819486 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:34.819486 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:34.819486 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:34.819486 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:34.819486 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:34.819486 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 24 08:28:34.819486 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 24 08:28:34.819486 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:34.819486 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:34.819486 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:28:34.819486 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:28:34.819486 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:28:34.819486 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:28:34.819486 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 24 08:28:34.819486 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 24 08:28:34.820589 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 24 08:28:34.820589 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 24 08:28:34.820589 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 24 08:28:34.820589 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 24 08:28:34.820589 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 24 08:28:34.820589 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:34.820589 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 476/968] 10.222.0.13 () {60 vars in 1131 bytes} [Fri Jun 24 08:28:34 2022] POST /identity/v3/users => generated 160 bytes in 189 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:34.820589 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466cb2da-a65a-408d-8c65-032e514e952a None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:34.837142 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e4d09b-ea49-467e-8036-36dfdc517325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:34.837735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e4d09b-ea49-467e-8036-36dfdc517325 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:34.838091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e4d09b-ea49-467e-8036-36dfdc517325 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:34.838367 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e4d09b-ea49-467e-8036-36dfdc517325 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:34.908327 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:34.916535 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e4d09b-ea49-467e-8036-36dfdc517325 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:34.918205 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e4d09b-ea49-467e-8036-36dfdc517325 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:34.933883 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466cb2da-a65a-408d-8c65-032e514e952a None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:34.970311 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:34.972051 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d16f4-f524-466d-82c7-c614c05ee707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:34.972788 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d16f4-f524-466d-82c7-c614c05ee707 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:34.973101 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d16f4-f524-466d-82c7-c614c05ee707 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:34.973372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d16f4-f524-466d-82c7-c614c05ee707 None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:34.974104 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6d16f4-f524-466d-82c7-c614c05ee707 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:34.974410 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6d16f4-f524-466d-82c7-c614c05ee707 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:34.975974 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6d16f4-f524-466d-82c7-c614c05ee707 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:34.984080 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 477/969] 10.222.0.13 () {56 vars in 1122 bytes} [Fri Jun 24 08:28:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:34.984859 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d16f4-f524-466d-82c7-c614c05ee707 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:35.035335 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 493/970] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:28:34 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 08:28:35.036573 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e4d09b-ea49-467e-8036-36dfdc517325 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:35.093954 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d16f4-f524-466d-82c7-c614c05ee707 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:35.128313 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ef1a5-4300-4277-a0cf-1349500288fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:35.129190 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ef1a5-4300-4277-a0cf-1349500288fc None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:35.129533 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ef1a5-4300-4277-a0cf-1349500288fc None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:35.129900 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ef1a5-4300-4277-a0cf-1349500288fc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:35.138764 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396ef1a5-4300-4277-a0cf-1349500288fc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:35.140283 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396ef1a5-4300-4277-a0cf-1349500288fc None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:35.151573 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e4d09b-ea49-467e-8036-36dfdc517325 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:35.155793 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-396ef1a5-4300-4277-a0cf-1349500288fc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 08:28:35.155793 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:35.155793 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:35.155793 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:35.155793 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:35.155793 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:35.155793 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:35.155793 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:35.155793 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:35.155793 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:35.155793 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:35.155793 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:35.155793 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:28:35.155793 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:28:35.155793 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:28:35.155793 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:28:35.156915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:35.156915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:35.156915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:28:35.156915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:28:35.156915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:28:35.156915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:28:35.156915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:35.156915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:35.156915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:35.156915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:35.156915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:35.156915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:35.156915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:35.156915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:35.156915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:35.156915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:35.156915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:35.156915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:35.158055 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:35.158055 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:35.158055 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:28:35.158055 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:28:35.158055 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:28:35.158055 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:28:35.158055 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:28:35.158055 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:28:35.158055 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 08:28:35.158055 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:35.158055 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 478/971] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:28:34 2022] GET /identity/v3/users/cinder => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:35.158055 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ef1a5-4300-4277-a0cf-1349500288fc None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:35.160488 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:35.188880 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba24d4-bb67-4091-8f49-7486cadb1407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:35.189540 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba24d4-bb67-4091-8f49-7486cadb1407 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:35.189858 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba24d4-bb67-4091-8f49-7486cadb1407 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:35.190136 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba24d4-bb67-4091-8f49-7486cadb1407 None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:35.190832 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ba24d4-bb67-4091-8f49-7486cadb1407 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:35.191134 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ba24d4-bb67-4091-8f49-7486cadb1407 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:35.192487 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ba24d4-bb67-4091-8f49-7486cadb1407 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:35.197408 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 494/972] 10.222.0.13 () {56 vars in 1122 bytes} [Fri Jun 24 08:28:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:35.198630 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4e939-97ac-4999-927c-d36a14ff2eb4 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:35.198917 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4e939-97ac-4999-927c-d36a14ff2eb4 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:35.199191 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4e939-97ac-4999-927c-d36a14ff2eb4 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:35.200580 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 495/973] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:35.201066 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d4e939-97ac-4999-927c-d36a14ff2eb4 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:35.217090 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:35.264400 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ef1a5-4300-4277-a0cf-1349500288fc None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:35.299431 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b847736b-ff71-4991-8c55-73c1b30096da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:35.300145 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b847736b-ff71-4991-8c55-73c1b30096da None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:35.300429 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b847736b-ff71-4991-8c55-73c1b30096da None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:35.300715 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b847736b-ff71-4991-8c55-73c1b30096da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:35.303782 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d4e939-97ac-4999-927c-d36a14ff2eb4 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:35.311275 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b847736b-ff71-4991-8c55-73c1b30096da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:35.312723 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b847736b-ff71-4991-8c55-73c1b30096da None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:35.330100 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-b847736b-ff71-4991-8c55-73c1b30096da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:35.330100 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:35.330100 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:35.330100 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:35.330100 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:35.330100 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:35.330100 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:35.330100 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:35.330100 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:35.330100 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:35.330100 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:35.330100 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:35.330100 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:28:35.330100 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:28:35.330100 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:28:35.331183 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:28:35.331183 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:35.331183 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:35.331183 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:35.331183 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:35.331183 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:35.331183 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:35.331183 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:35.331183 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:35.331183 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:35.331183 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:35.331183 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:35.331183 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:35.331183 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:35.331183 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:35.331183 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:35.331183 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:35.332066 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:28:35.332066 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:28:35.332066 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:35.332066 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:35.332066 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 479/974] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:35.332066 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b847736b-ff71-4991-8c55-73c1b30096da None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:35.345824 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487a5e0c-0fce-4f43-b05c-ef27283291c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:35.346494 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a5e0c-0fce-4f43-b05c-ef27283291c9 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:35.346778 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a5e0c-0fce-4f43-b05c-ef27283291c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:35.347055 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a5e0c-0fce-4f43-b05c-ef27283291c9 None admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:35.347899 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487a5e0c-0fce-4f43-b05c-ef27283291c9 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:35.348206 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487a5e0c-0fce-4f43-b05c-ef27283291c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:35.349627 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487a5e0c-0fce-4f43-b05c-ef27283291c9 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:35.388203 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 496/975] 10.222.0.13 () {56 vars in 1101 bytes} [Fri Jun 24 08:28:35 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:35.389468 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878d805-8335-41ae-b5e9-987e530ad5e1 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:35.389784 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878d805-8335-41ae-b5e9-987e530ad5e1 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:35.390064 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878d805-8335-41ae-b5e9-987e530ad5e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:35.402324 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:35.412741 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-7878d805-8335-41ae-b5e9-987e530ad5e1 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:35.414007 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-7878d805-8335-41ae-b5e9-987e530ad5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:35.448865 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b847736b-ff71-4991-8c55-73c1b30096da None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:35.491546 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 497/976] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:35 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:28:35.492420 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878d805-8335-41ae-b5e9-987e530ad5e1 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:35.495866 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:35.507055 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f39c29-7085-4b20-9a1d-f4afe9d60030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:35.508102 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f39c29-7085-4b20-9a1d-f4afe9d60030 None admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:35.509107 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f39c29-7085-4b20-9a1d-f4afe9d60030 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:35.509606 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f39c29-7085-4b20-9a1d-f4afe9d60030 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:35.510575 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f39c29-7085-4b20-9a1d-f4afe9d60030 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:35.512514 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f39c29-7085-4b20-9a1d-f4afe9d60030 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:35.537125 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 480/977] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Jun 24 08:28:35 2022] POST /identity/v3/endpoints => generated 349 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:28:35.537125 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f39c29-7085-4b20-9a1d-f4afe9d60030 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:35.630865 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878d805-8335-41ae-b5e9-987e530ad5e1 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:35.680786 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839b842a-3a17-4549-a773-6801892013c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:35.681806 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b842a-3a17-4549-a773-6801892013c8 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:35.682243 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b842a-3a17-4549-a773-6801892013c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:35.682670 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b842a-3a17-4549-a773-6801892013c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:35.683554 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839b842a-3a17-4549-a773-6801892013c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:35.684011 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839b842a-3a17-4549-a773-6801892013c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:35.685595 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839b842a-3a17-4549-a773-6801892013c8 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:35.686109 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f39c29-7085-4b20-9a1d-f4afe9d60030 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:35.694944 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:35.704824 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 498/978] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:28:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:35.712951 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839b842a-3a17-4549-a773-6801892013c8 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:35.731383 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51285ff-3693-4e41-8a5a-1f69ab51fcda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:35.732048 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51285ff-3693-4e41-8a5a-1f69ab51fcda None admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:35.732330 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51285ff-3693-4e41-8a5a-1f69ab51fcda None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:35.732601 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51285ff-3693-4e41-8a5a-1f69ab51fcda None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:35.733331 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51285ff-3693-4e41-8a5a-1f69ab51fcda None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:35.734795 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51285ff-3693-4e41-8a5a-1f69ab51fcda None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:35.751616 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 481/979] 10.222.0.13 () {60 vars in 1141 bytes} [Fri Jun 24 08:28:35 2022] POST /identity/v3/services => generated 242 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:28:35.789192 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:35.842699 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839b842a-3a17-4549-a773-6801892013c8 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:35.891227 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3c88ef-b093-4205-ae7b-0a4ad3f80fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:35.891959 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c88ef-b093-4205-ae7b-0a4ad3f80fc4 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:35.892243 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c88ef-b093-4205-ae7b-0a4ad3f80fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:35.892519 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c88ef-b093-4205-ae7b-0a4ad3f80fc4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:35.901037 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3c88ef-b093-4205-ae7b-0a4ad3f80fc4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:35.902535 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3c88ef-b093-4205-ae7b-0a4ad3f80fc4 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:35.917465 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-3c3c88ef-b093-4205-ae7b-0a4ad3f80fc4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:28:35.917465 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:35.917465 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:35.917465 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:35.917465 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:35.917465 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:35.917465 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:35.917465 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:35.917465 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:35.917465 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:35.917465 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:35.917465 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:35.917465 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:28:35.917465 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:28:35.917465 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:28:35.918651 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:28:35.918651 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:35.918651 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:35.918651 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:35.918651 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:35.918651 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:35.918651 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:35.918651 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:35.918651 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:35.918651 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:35.918651 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:35.918651 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:35.918651 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:35.918651 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:35.918651 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:35.918651 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:35.918651 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:35.919660 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:28:35.919660 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:28:35.919660 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:28:35.919660 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:28:35.919660 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:28:35.919660 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:28:35.919660 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:28:35.919660 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:35.919660 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 499/980] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:28:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:28:35.924842 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51285ff-3693-4e41-8a5a-1f69ab51fcda None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:36.038830 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51285ff-3693-4e41-8a5a-1f69ab51fcda None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:36.074099 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbde44b-f8b3-4837-a36f-e9247a7df720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:36.074767 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbde44b-f8b3-4837-a36f-e9247a7df720 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:36.075054 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbde44b-f8b3-4837-a36f-e9247a7df720 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:36.075329 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbde44b-f8b3-4837-a36f-e9247a7df720 None admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:36.076110 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbde44b-f8b3-4837-a36f-e9247a7df720 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:36.076420 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbde44b-f8b3-4837-a36f-e9247a7df720 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:36.077878 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbde44b-f8b3-4837-a36f-e9247a7df720 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:36.095569 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 482/981] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Jun 24 08:28:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:36.096979 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3c88ef-b093-4205-ae7b-0a4ad3f80fc4 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:36.200900 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3c88ef-b093-4205-ae7b-0a4ad3f80fc4 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:36.215943 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:36.238204 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fe888b-48a7-4d82-9962-8856c4147a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:36.238923 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fe888b-48a7-4d82-9962-8856c4147a5b None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:36.239211 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fe888b-48a7-4d82-9962-8856c4147a5b None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:36.239491 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fe888b-48a7-4d82-9962-8856c4147a5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:36.240368 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fe888b-48a7-4d82-9962-8856c4147a5b None admin] RBAC: Adding query filter params (role.id=f9658866044e4fdeaddcc91781166faf, scope.project.id=180b0af5863349e99ae412b07c1e9465, user.id=85d9c4c57681447c898ebfd7990b05b6) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:36.240686 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fe888b-48a7-4d82-9962-8856c4147a5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:36.242135 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fe888b-48a7-4d82-9962-8856c4147a5b None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:36.257870 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 500/982] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Jun 24 08:28:36 2022] GET /identity/v3/role_assignments?user.id=85d9c4c57681447c898ebfd7990b05b6&scope.project.id=180b0af5863349e99ae412b07c1e9465&role.id=f9658866044e4fdeaddcc91781166faf => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:37.613395 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2377d7f-133c-4eb2-b406-fdb9dee635ed None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:37.613949 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2377d7f-133c-4eb2-b406-fdb9dee635ed None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:37.613949 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2377d7f-133c-4eb2-b406-fdb9dee635ed None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:37.615372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 483/983] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:37.621192 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d19bf8-08fc-43b2-9a5b-965fa76acf49 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:37.621619 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d19bf8-08fc-43b2-9a5b-965fa76acf49 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:37.621822 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d19bf8-08fc-43b2-9a5b-965fa76acf49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:37.639268 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-86d19bf8-08fc-43b2-9a5b-965fa76acf49 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:37.640376 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-86d19bf8-08fc-43b2-9a5b-965fa76acf49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:37.708890 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 501/984] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:37 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:28:37.731462 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2377d7f-133c-4eb2-b406-fdb9dee635ed None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:37.759785 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:37.838774 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2377d7f-133c-4eb2-b406-fdb9dee635ed None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:37.862647 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2bb0ec-1319-4575-9d2d-4bc87924879b None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:37.862944 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2bb0ec-1319-4575-9d2d-4bc87924879b None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:37.863194 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2bb0ec-1319-4575-9d2d-4bc87924879b None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:37.864802 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 502/985] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:28:37.870192 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534a64f-4b07-4ece-981c-2bfff00de0d2 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:37.870480 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534a64f-4b07-4ece-981c-2bfff00de0d2 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:37.870757 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534a64f-4b07-4ece-981c-2bfff00de0d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:37.874231 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941d770e-e990-4f4a-85f3-85a6f8e846fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:37.875034 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941d770e-e990-4f4a-85f3-85a6f8e846fa None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:37.875313 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941d770e-e990-4f4a-85f3-85a6f8e846fa None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:37.875582 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941d770e-e990-4f4a-85f3-85a6f8e846fa None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:37.876225 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941d770e-e990-4f4a-85f3-85a6f8e846fa None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:37.877845 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941d770e-e990-4f4a-85f3-85a6f8e846fa None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:37.890027 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-a534a64f-4b07-4ece-981c-2bfff00de0d2 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:37.890806 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-941d770e-e990-4f4a-85f3-85a6f8e846fa None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 08:28:37.890806 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:37.890806 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 08:28:37.890806 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 08:28:37.890806 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 08:28:37.890806 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 08:28:37.890806 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 08:28:37.890806 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:28:37.890806 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 08:28:37.890806 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:37.890806 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:28:37.890806 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:37.890806 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:37.890806 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:37.890806 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:37.890806 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:37.890806 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:37.892430 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:37.892430 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:37.892430 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:37.892430 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:37.892430 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:37.892430 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:37.892430 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 08:28:37.892430 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 08:28:37.892430 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 08:28:37.892430 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 08:28:37.892430 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:37.892430 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:37.892430 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:37.892430 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:37.892430 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:37.892430 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:37.894100 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:37.894100 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:37.894100 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:37.894100 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:37.894100 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:37.894100 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:37.894100 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:37.894100 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:37.894100 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:37.894100 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:37.894100 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 08:28:37.894100 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:28:37.894100 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 08:28:37.894100 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:37.894100 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 484/986] 10.222.0.13 () {56 vars in 1126 bytes} [Fri Jun 24 08:28:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:37.895530 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-a534a64f-4b07-4ece-981c-2bfff00de0d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:37.898336 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941d770e-e990-4f4a-85f3-85a6f8e846fa None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:37.977857 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 503/987] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:37 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 08:28:37.990971 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:37.993848 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a534a64f-4b07-4ece-981c-2bfff00de0d2 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:38.004362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941d770e-e990-4f4a-85f3-85a6f8e846fa None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:38.037035 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:38.040917 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068ead72-c5d2-45fd-b95b-df55a68ac6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:38.041583 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ead72-c5d2-45fd-b95b-df55a68ac6b5 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:38.041896 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ead72-c5d2-45fd-b95b-df55a68ac6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:38.042176 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ead72-c5d2-45fd-b95b-df55a68ac6b5 None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:38.042882 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068ead72-c5d2-45fd-b95b-df55a68ac6b5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:38.043191 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068ead72-c5d2-45fd-b95b-df55a68ac6b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:38.044568 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068ead72-c5d2-45fd-b95b-df55a68ac6b5 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:38.054139 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 485/988] 10.222.0.13 () {56 vars in 1122 bytes} [Fri Jun 24 08:28:37 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:38.055120 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068ead72-c5d2-45fd-b95b-df55a68ac6b5 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:38.107859 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a534a64f-4b07-4ece-981c-2bfff00de0d2 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:38.145046 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e69218a-545d-47fd-b8ed-f516ed09f39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:38.145777 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69218a-545d-47fd-b8ed-f516ed09f39c None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:38.146065 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69218a-545d-47fd-b8ed-f516ed09f39c None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:38.146340 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69218a-545d-47fd-b8ed-f516ed09f39c None admin] PATH_INFO: `/v3/services/compute` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:38.146992 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e69218a-545d-47fd-b8ed-f516ed09f39c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:38.148362 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e69218a-545d-47fd-b8ed-f516ed09f39c None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:38.160059 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-0e69218a-545d-47fd-b8ed-f516ed09f39c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 08:28:38.160059 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:38.160059 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 08:28:38.160059 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 08:28:38.160059 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 08:28:38.160059 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 08:28:38.160059 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 08:28:38.160059 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:28:38.160059 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 08:28:38.160059 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:38.160059 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:28:38.160059 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:38.160059 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:38.160059 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:38.160059 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:38.160059 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:38.160059 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:38.161046 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:38.161046 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:38.161046 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:38.161046 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:38.161046 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:38.161046 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:38.161046 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 08:28:38.161046 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 08:28:38.161046 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 08:28:38.161046 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 08:28:38.161046 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:38.161046 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:38.161046 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:38.161046 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:38.161046 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:38.161046 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:38.162045 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:38.162045 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:38.162045 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:38.162045 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:38.162045 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:38.162045 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:38.162045 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:38.162045 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:38.162045 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:38.162045 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:38.162045 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 08:28:38.162045 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:28:38.162045 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 08:28:38.162045 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:38.162045 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 504/989] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:28:37 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:38.167070 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e69218a-545d-47fd-b8ed-f516ed09f39c None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:38.170436 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068ead72-c5d2-45fd-b95b-df55a68ac6b5 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:38.207588 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa658fec-0f31-41f8-8143-3ca38c8ba42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:38.208253 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa658fec-0f31-41f8-8143-3ca38c8ba42b None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:38.208535 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa658fec-0f31-41f8-8143-3ca38c8ba42b None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:38.208814 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa658fec-0f31-41f8-8143-3ca38c8ba42b None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:38.209509 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa658fec-0f31-41f8-8143-3ca38c8ba42b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:38.209848 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa658fec-0f31-41f8-8143-3ca38c8ba42b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:38.211216 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa658fec-0f31-41f8-8143-3ca38c8ba42b None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:38.216323 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 486/990] 10.222.0.13 () {56 vars in 1122 bytes} [Fri Jun 24 08:28:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:38.222002 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa658fec-0f31-41f8-8143-3ca38c8ba42b None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:38.244087 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:38.282177 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e69218a-545d-47fd-b8ed-f516ed09f39c None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:38.283849 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:38.320551 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef824f2-7e4d-4529-b5ae-2baf1120bd78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:38.322096 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef824f2-7e4d-4529-b5ae-2baf1120bd78 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:38.322243 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef824f2-7e4d-4529-b5ae-2baf1120bd78 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:38.322441 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef824f2-7e4d-4529-b5ae-2baf1120bd78 None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:38.323146 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef824f2-7e4d-4529-b5ae-2baf1120bd78 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:38.323452 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef824f2-7e4d-4529-b5ae-2baf1120bd78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:38.324822 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef824f2-7e4d-4529-b5ae-2baf1120bd78 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:38.333715 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 505/991] 10.222.0.13 () {56 vars in 1112 bytes} [Fri Jun 24 08:28:38 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:38.335564 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef824f2-7e4d-4529-b5ae-2baf1120bd78 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:38.340963 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa658fec-0f31-41f8-8143-3ca38c8ba42b None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:38.378447 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322b7695-230c-4d01-b4cf-45f82e0d300b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:38.379113 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b7695-230c-4d01-b4cf-45f82e0d300b None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:38.379403 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b7695-230c-4d01-b4cf-45f82e0d300b None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:38.379684 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b7695-230c-4d01-b4cf-45f82e0d300b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:38.380481 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322b7695-230c-4d01-b4cf-45f82e0d300b None admin] RBAC: Adding query filter params (interface=admin, service_id=ce4106318b154293be53a8b74d6bf50f, region_id=RegionOne) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:38.380796 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322b7695-230c-4d01-b4cf-45f82e0d300b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:38.382220 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322b7695-230c-4d01-b4cf-45f82e0d300b None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:38.390305 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 487/992] 10.222.0.13 () {56 vars in 1249 bytes} [Fri Jun 24 08:28:38 2022] GET /identity/v3/endpoints?service_id=ce4106318b154293be53a8b74d6bf50f&interface=admin®ion_id=RegionOne => generated 193 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:38.391602 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3821d8cc-754f-4477-a251-44637bc36a73 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:38.391897 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3821d8cc-754f-4477-a251-44637bc36a73 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:38.392175 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3821d8cc-754f-4477-a251-44637bc36a73 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:38.397640 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 488/993] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:38.398369 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3821d8cc-754f-4477-a251-44637bc36a73 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:38.443010 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef824f2-7e4d-4529-b5ae-2baf1120bd78 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:38.485484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b0085-3375-49ba-b844-de005ec426a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:38.485824 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:38.486186 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b0085-3375-49ba-b844-de005ec426a3 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:38.486472 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b0085-3375-49ba-b844-de005ec426a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:38.486794 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b0085-3375-49ba-b844-de005ec426a3 None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:38.487766 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178b0085-3375-49ba-b844-de005ec426a3 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:38.488102 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178b0085-3375-49ba-b844-de005ec426a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:38.489534 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178b0085-3375-49ba-b844-de005ec426a3 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:38.495410 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 506/994] 10.222.0.13 () {56 vars in 1112 bytes} [Fri Jun 24 08:28:38 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:38.496520 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc2beaf-c5d7-49ce-b3b7-78c8fa2a3872 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:38.496811 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc2beaf-c5d7-49ce-b3b7-78c8fa2a3872 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:38.497105 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc2beaf-c5d7-49ce-b3b7-78c8fa2a3872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:38.508201 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3821d8cc-754f-4477-a251-44637bc36a73 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:38.516389 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-bcc2beaf-c5d7-49ce-b3b7-78c8fa2a3872 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:38.517601 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-bcc2beaf-c5d7-49ce-b3b7-78c8fa2a3872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:38.543122 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:38.543277 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d48f22-a083-43d5-b135-80d8e2b1ccfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:38.543869 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d48f22-a083-43d5-b135-80d8e2b1ccfb None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:38.544405 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d48f22-a083-43d5-b135-80d8e2b1ccfb None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:38.544405 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d48f22-a083-43d5-b135-80d8e2b1ccfb None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:38.545104 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d48f22-a083-43d5-b135-80d8e2b1ccfb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:38.546488 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d48f22-a083-43d5-b135-80d8e2b1ccfb None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:38.552650 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 489/995] 10.222.0.13 () {56 vars in 1087 bytes} [Fri Jun 24 08:28:38 2022] GET /identity/v3/services => generated 1727 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:28:38.553415 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d48f22-a083-43d5-b135-80d8e2b1ccfb None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:38.593471 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 507/996] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:38 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:28:38.616601 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc2beaf-c5d7-49ce-b3b7-78c8fa2a3872 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:38.675571 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d48f22-a083-43d5-b135-80d8e2b1ccfb None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:38.718379 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2438b406-c78f-494d-8d52-ac1b5fc832f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:38.719050 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438b406-c78f-494d-8d52-ac1b5fc832f8 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:38.719340 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438b406-c78f-494d-8d52-ac1b5fc832f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:38.719626 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438b406-c78f-494d-8d52-ac1b5fc832f8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:38.720416 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2438b406-c78f-494d-8d52-ac1b5fc832f8 None admin] RBAC: Adding query filter params (interface=public, service_id=74e30381f9e44f05b801d72e3eb35911, region_id=RegionOne) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:38.720735 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2438b406-c78f-494d-8d52-ac1b5fc832f8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:38.722321 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2438b406-c78f-494d-8d52-ac1b5fc832f8 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:38.729319 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 490/997] 10.222.0.13 () {56 vars in 1251 bytes} [Fri Jun 24 08:28:38 2022] GET /identity/v3/endpoints?service_id=74e30381f9e44f05b801d72e3eb35911&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:38.734791 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2438b406-c78f-494d-8d52-ac1b5fc832f8 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:38.741801 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc2beaf-c5d7-49ce-b3b7-78c8fa2a3872 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:38.749303 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:38.780267 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc43be08-a2d0-4bd8-9749-a49d23923ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:38.781074 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43be08-a2d0-4bd8-9749-a49d23923ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:38.781360 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43be08-a2d0-4bd8-9749-a49d23923ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:38.781638 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43be08-a2d0-4bd8-9749-a49d23923ce3 None admin] PATH_INFO: `/v3/services/image` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:38.782730 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc43be08-a2d0-4bd8-9749-a49d23923ce3 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:38.784148 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc43be08-a2d0-4bd8-9749-a49d23923ce3 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:38.802483 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-dc43be08-a2d0-4bd8-9749-a49d23923ce3 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 08:28:38.802483 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:38.802483 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 08:28:38.802483 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 08:28:38.802483 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 08:28:38.802483 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 08:28:38.802483 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 08:28:38.802483 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:28:38.802483 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 08:28:38.802483 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:38.802483 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:28:38.802483 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:38.802483 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:38.802483 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:38.802483 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:38.802483 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:38.802483 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:38.803555 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:38.803555 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:38.803555 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:38.803555 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:38.803555 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:38.803555 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:38.803555 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 08:28:38.803555 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 08:28:38.803555 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 08:28:38.803555 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 08:28:38.803555 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:38.803555 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:38.803555 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:38.803555 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:38.803555 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:38.803555 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:38.804405 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:38.804405 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:38.804405 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:38.804405 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:38.804405 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:38.804405 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:38.804405 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:38.804405 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:38.804405 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:38.804405 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:38.804405 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 08:28:38.804405 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:28:38.804405 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 08:28:38.804405 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:38.804405 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 508/998] 10.222.0.13 () {56 vars in 1105 bytes} [Fri Jun 24 08:28:38 2022] GET /identity/v3/services/image => generated 86 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:28:38.811308 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc43be08-a2d0-4bd8-9749-a49d23923ce3 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:38.842852 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:38.852329 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2438b406-c78f-494d-8d52-ac1b5fc832f8 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:38.889833 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd478aa-1b15-41a5-b563-c52e0384e7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:38.890518 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd478aa-1b15-41a5-b563-c52e0384e7bc None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:38.890729 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd478aa-1b15-41a5-b563-c52e0384e7bc None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:38.891003 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd478aa-1b15-41a5-b563-c52e0384e7bc None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:38.891682 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd478aa-1b15-41a5-b563-c52e0384e7bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:38.893049 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd478aa-1b15-41a5-b563-c52e0384e7bc None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:38.909726 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 491/999] 10.222.0.13 () {56 vars in 1087 bytes} [Fri Jun 24 08:28:38 2022] GET /identity/v3/services => generated 1727 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:28:38.941801 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc43be08-a2d0-4bd8-9749-a49d23923ce3 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:38.980591 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fd276f-ac45-488e-b9d1-86a2dd5d642d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:38.981280 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd276f-ac45-488e-b9d1-86a2dd5d642d None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:38.981571 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd276f-ac45-488e-b9d1-86a2dd5d642d None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:38.981885 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd276f-ac45-488e-b9d1-86a2dd5d642d None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:38.982611 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fd276f-ac45-488e-b9d1-86a2dd5d642d None admin] RBAC: Adding query filter params (name=image) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:38.982923 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fd276f-ac45-488e-b9d1-86a2dd5d642d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:38.984325 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fd276f-ac45-488e-b9d1-86a2dd5d642d None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:38.989943 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 509/1000] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:38 2022] GET /identity/v3/services?name=image => generated 122 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:38.997126 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd478aa-1b15-41a5-b563-c52e0384e7bc None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:39.082726 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:39.110902 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd478aa-1b15-41a5-b563-c52e0384e7bc None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:39.147682 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c04c29-2426-4636-9aae-03e79e5f63fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:39.148444 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c04c29-2426-4636-9aae-03e79e5f63fa None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:39.148797 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c04c29-2426-4636-9aae-03e79e5f63fa None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:39.149189 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c04c29-2426-4636-9aae-03e79e5f63fa None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:39.150103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c04c29-2426-4636-9aae-03e79e5f63fa None admin] RBAC: Adding query filter params (type=image) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:39.150473 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c04c29-2426-4636-9aae-03e79e5f63fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:39.151925 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c04c29-2426-4636-9aae-03e79e5f63fa None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:39.161728 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 492/1001] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:38 2022] GET /identity/v3/services?type=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:40.651399 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a029c-1dce-416d-a568-29476c8e2077 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:40.651894 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a029c-1dce-416d-a568-29476c8e2077 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:40.651894 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a029c-1dce-416d-a568-29476c8e2077 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:40.653392 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 510/1002] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:40.658658 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370bce47-cb0b-4044-bf03-92467488b6c0 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:40.659082 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370bce47-cb0b-4044-bf03-92467488b6c0 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:40.659247 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370bce47-cb0b-4044-bf03-92467488b6c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:40.676541 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-370bce47-cb0b-4044-bf03-92467488b6c0 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:40.678096 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-370bce47-cb0b-4044-bf03-92467488b6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:40.750309 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 493/1003] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:40 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:28:40.773912 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5a029c-1dce-416d-a568-29476c8e2077 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:40.867153 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:40.881500 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5a029c-1dce-416d-a568-29476c8e2077 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:40.919385 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a6de3b-8a68-4502-96b1-cced2f6995ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:40.920202 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a6de3b-8a68-4502-96b1-cced2f6995ec None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:40.920487 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a6de3b-8a68-4502-96b1-cced2f6995ec None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:40.920764 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a6de3b-8a68-4502-96b1-cced2f6995ec None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:40.921413 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a6de3b-8a68-4502-96b1-cced2f6995ec None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:40.922813 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a6de3b-8a68-4502-96b1-cced2f6995ec None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:40.934874 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-d8a6de3b-8a68-4502-96b1-cced2f6995ec None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 08:28:40.934874 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:40.934874 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 08:28:40.934874 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 08:28:40.934874 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 08:28:40.934874 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 08:28:40.934874 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 08:28:40.934874 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:28:40.934874 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 08:28:40.934874 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:40.934874 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:28:40.934874 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:40.934874 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:40.934874 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:40.934874 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:40.934874 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:40.934874 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:40.935897 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:40.935897 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:40.935897 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:40.935897 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:40.935897 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:40.935897 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:40.935897 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 08:28:40.935897 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 08:28:40.935897 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 08:28:40.935897 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 08:28:40.935897 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:40.935897 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:40.935897 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:40.935897 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:40.935897 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:40.935897 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:40.936840 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:40.936840 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:40.936840 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:40.936840 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:40.936840 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:40.936840 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:40.936840 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:40.936840 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:40.936840 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:40.936840 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:40.936840 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 08:28:40.936840 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:28:40.936840 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 08:28:40.936840 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:40.940945 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 511/1004] 10.222.0.13 () {56 vars in 1126 bytes} [Fri Jun 24 08:28:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:40.942307 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370bce47-cb0b-4044-bf03-92467488b6c0 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:41.011348 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:41.037759 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67bdd30-73dd-4eab-88dc-6ace9fbc46f6 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:41.038114 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67bdd30-73dd-4eab-88dc-6ace9fbc46f6 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:41.038468 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67bdd30-73dd-4eab-88dc-6ace9fbc46f6 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:41.040683 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 512/1005] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:41.046382 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b069f-1874-4fef-8aab-1d7a8f706563 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:41.046704 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b069f-1874-4fef-8aab-1d7a8f706563 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:41.047131 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b069f-1874-4fef-8aab-1d7a8f706563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:41.055796 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370bce47-cb0b-4044-bf03-92467488b6c0 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:41.067734 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-a62b069f-1874-4fef-8aab-1d7a8f706563 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:41.069018 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-a62b069f-1874-4fef-8aab-1d7a8f706563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:41.095843 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1b221e-a623-476c-be90-eed6fb076ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:41.096518 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b221e-a623-476c-be90-eed6fb076ede None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:41.096804 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b221e-a623-476c-be90-eed6fb076ede None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:41.097078 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b221e-a623-476c-be90-eed6fb076ede None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:41.097818 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1b221e-a623-476c-be90-eed6fb076ede None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:41.098127 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1b221e-a623-476c-be90-eed6fb076ede None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:41.099596 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1b221e-a623-476c-be90-eed6fb076ede None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:41.105739 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 494/1006] 10.222.0.13 () {56 vars in 1122 bytes} [Fri Jun 24 08:28:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:41.110938 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1b221e-a623-476c-be90-eed6fb076ede None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:41.144386 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 513/1007] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:41 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 08:28:41.167705 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62b069f-1874-4fef-8aab-1d7a8f706563 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:41.222690 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1b221e-a623-476c-be90-eed6fb076ede None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:41.254029 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:41.255055 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:41.260898 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb215ed-7a02-4a83-9648-afa23dae98de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:41.261558 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb215ed-7a02-4a83-9648-afa23dae98de None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:41.261886 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb215ed-7a02-4a83-9648-afa23dae98de None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:41.262164 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb215ed-7a02-4a83-9648-afa23dae98de None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:41.262866 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb215ed-7a02-4a83-9648-afa23dae98de None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:41.263171 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb215ed-7a02-4a83-9648-afa23dae98de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:41.264546 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb215ed-7a02-4a83-9648-afa23dae98de None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:41.269340 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 495/1008] 10.222.0.13 () {56 vars in 1122 bytes} [Fri Jun 24 08:28:41 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:41.270767 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c97b5-38c4-4294-a914-74f0576f68f8 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:41.271058 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c97b5-38c4-4294-a914-74f0576f68f8 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:41.271338 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c97b5-38c4-4294-a914-74f0576f68f8 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:41.272569 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 496/1009] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:41.275485 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27c97b5-38c4-4294-a914-74f0576f68f8 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:41.278104 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62b069f-1874-4fef-8aab-1d7a8f706563 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:41.314848 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e4b4f9-a1b5-414a-a550-f6a3e3daf54c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:41.315633 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e4b4f9-a1b5-414a-a550-f6a3e3daf54c None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:41.315920 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e4b4f9-a1b5-414a-a550-f6a3e3daf54c None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:41.316197 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e4b4f9-a1b5-414a-a550-f6a3e3daf54c None admin] PATH_INFO: `/v3/services/compute` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:41.316848 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e4b4f9-a1b5-414a-a550-f6a3e3daf54c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:41.318257 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e4b4f9-a1b5-414a-a550-f6a3e3daf54c None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:41.330431 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-17e4b4f9-a1b5-414a-a550-f6a3e3daf54c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 08:28:41.330431 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:41.330431 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 08:28:41.330431 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 08:28:41.330431 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 08:28:41.330431 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 08:28:41.330431 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 08:28:41.330431 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:28:41.330431 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 08:28:41.330431 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:41.330431 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:28:41.330431 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:41.330431 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:41.330431 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:41.330431 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:41.330431 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:41.330431 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:41.331602 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:41.331602 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:41.331602 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:41.331602 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:41.331602 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:41.331602 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:41.331602 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 08:28:41.331602 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 08:28:41.331602 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 08:28:41.331602 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 08:28:41.331602 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:41.331602 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:41.331602 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:41.331602 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:41.331602 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:41.331602 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:41.332447 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:41.332447 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:41.332447 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:41.332447 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:41.332447 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:41.332447 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:41.332447 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:41.332447 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:41.332447 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:41.332447 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:41.332447 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 08:28:41.332447 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:28:41.332447 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 08:28:41.332447 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:41.332447 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 514/1010] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:28:41 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:41.334980 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaef017-b32b-422d-af19-1f0257782cf7 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:41.335267 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaef017-b32b-422d-af19-1f0257782cf7 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:41.335544 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaef017-b32b-422d-af19-1f0257782cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:41.352921 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-daaef017-b32b-422d-af19-1f0257782cf7 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:41.354476 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-daaef017-b32b-422d-af19-1f0257782cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:41.381332 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27c97b5-38c4-4294-a914-74f0576f68f8 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:41.415656 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b8ce7-90fa-460b-8700-d51333d179c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:41.416295 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b8ce7-90fa-460b-8700-d51333d179c3 None admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:41.416586 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b8ce7-90fa-460b-8700-d51333d179c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:41.416837 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b8ce7-90fa-460b-8700-d51333d179c3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:41.417559 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240b8ce7-90fa-460b-8700-d51333d179c3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:41.418953 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240b8ce7-90fa-460b-8700-d51333d179c3 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:41.421103 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 515/1011] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:41 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:28:41.421959 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaef017-b32b-422d-af19-1f0257782cf7 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:41.460932 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 497/1012] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Jun 24 08:28:41 2022] POST /identity/v3/endpoints => generated 325 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:28:41.460932 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b8ce7-90fa-460b-8700-d51333d179c3 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:41.518945 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:41.521816 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:41.548036 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaef017-b32b-422d-af19-1f0257782cf7 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:41.583616 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b8ce7-90fa-460b-8700-d51333d179c3 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:41.588815 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea75da-12ef-4fe5-b6fc-8e8dbf592b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:41.589500 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea75da-12ef-4fe5-b6fc-8e8dbf592b95 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:41.589812 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea75da-12ef-4fe5-b6fc-8e8dbf592b95 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:41.590086 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea75da-12ef-4fe5-b6fc-8e8dbf592b95 None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:41.590922 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ea75da-12ef-4fe5-b6fc-8e8dbf592b95 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:41.591226 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ea75da-12ef-4fe5-b6fc-8e8dbf592b95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:41.593170 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ea75da-12ef-4fe5-b6fc-8e8dbf592b95 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:41.604079 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 516/1013] 10.222.0.13 () {56 vars in 1112 bytes} [Fri Jun 24 08:28:41 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:41.604381 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea75da-12ef-4fe5-b6fc-8e8dbf592b95 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:41.626247 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f18b429-64e2-45bb-8f11-e1fb12d13bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:41.626465 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18b429-64e2-45bb-8f11-e1fb12d13bb6 None admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:41.626649 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18b429-64e2-45bb-8f11-e1fb12d13bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:41.626934 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18b429-64e2-45bb-8f11-e1fb12d13bb6 None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:41.628305 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f18b429-64e2-45bb-8f11-e1fb12d13bb6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:41.629211 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f18b429-64e2-45bb-8f11-e1fb12d13bb6 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:41.647536 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 498/1014] 10.222.0.13 () {60 vars in 1141 bytes} [Fri Jun 24 08:28:41 2022] POST /identity/v3/services => generated 242 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:28:41.729604 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea75da-12ef-4fe5-b6fc-8e8dbf592b95 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:41.773629 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40842d9-76c0-425e-a532-ea31acd4d11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:41.774769 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40842d9-76c0-425e-a532-ea31acd4d11f None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:41.775055 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40842d9-76c0-425e-a532-ea31acd4d11f None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:41.775327 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40842d9-76c0-425e-a532-ea31acd4d11f None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:41.776045 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40842d9-76c0-425e-a532-ea31acd4d11f None admin] RBAC: Adding query filter params (type=compute) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:41.776349 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40842d9-76c0-425e-a532-ea31acd4d11f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:41.777743 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40842d9-76c0-425e-a532-ea31acd4d11f None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:41.780221 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:41.789298 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 517/1015] 10.222.0.13 () {56 vars in 1112 bytes} [Fri Jun 24 08:28:41 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:41.791613 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f18b429-64e2-45bb-8f11-e1fb12d13bb6 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:41.904586 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f18b429-64e2-45bb-8f11-e1fb12d13bb6 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:41.919210 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:41.943886 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f89eb1-8ebd-438f-bb54-437cc96e4ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:41.944544 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f89eb1-8ebd-438f-bb54-437cc96e4ecd None admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:41.944826 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f89eb1-8ebd-438f-bb54-437cc96e4ecd None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:41.945100 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f89eb1-8ebd-438f-bb54-437cc96e4ecd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:41.946952 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f89eb1-8ebd-438f-bb54-437cc96e4ecd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:41.947973 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f89eb1-8ebd-438f-bb54-437cc96e4ecd None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:41.968297 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 499/1016] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Jun 24 08:28:41 2022] POST /identity/v3/endpoints => generated 334 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:28:43.562366 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cfabe0-4c23-4f93-979a-3fc012a20d51 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:43.562915 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cfabe0-4c23-4f93-979a-3fc012a20d51 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:43.562915 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cfabe0-4c23-4f93-979a-3fc012a20d51 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:43.564352 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 518/1017] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:43.569582 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895612f2-60c2-4495-94f1-0a2836d2b7d6 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:43.570005 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895612f2-60c2-4495-94f1-0a2836d2b7d6 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:43.570209 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895612f2-60c2-4495-94f1-0a2836d2b7d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:43.587148 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-895612f2-60c2-4495-94f1-0a2836d2b7d6 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:43.588350 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-895612f2-60c2-4495-94f1-0a2836d2b7d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:43.666573 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 500/1018] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:43 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:28:43.691055 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cfabe0-4c23-4f93-979a-3fc012a20d51 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:43.739535 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65621a58-4381-488e-b678-a5ff261cdd36 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:43.739841 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65621a58-4381-488e-b678-a5ff261cdd36 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:43.740091 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65621a58-4381-488e-b678-a5ff261cdd36 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:43.741590 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 501/1019] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:43.746890 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b8cac-07c7-4d68-b4cc-d8be7fcb46f3 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:43.747175 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b8cac-07c7-4d68-b4cc-d8be7fcb46f3 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:43.747452 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b8cac-07c7-4d68-b4cc-d8be7fcb46f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:43.764453 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-4f4b8cac-07c7-4d68-b4cc-d8be7fcb46f3 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:43.765629 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-4f4b8cac-07c7-4d68-b4cc-d8be7fcb46f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:43.801860 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cfabe0-4c23-4f93-979a-3fc012a20d51 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:43.840036 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84602de-cb52-4c88-bb75-f8e6645ffd07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:43.840876 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84602de-cb52-4c88-bb75-f8e6645ffd07 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:43.841165 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84602de-cb52-4c88-bb75-f8e6645ffd07 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:43.841446 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84602de-cb52-4c88-bb75-f8e6645ffd07 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:43.842803 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84602de-cb52-4c88-bb75-f8e6645ffd07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:43.844325 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84602de-cb52-4c88-bb75-f8e6645ffd07 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:43.851056 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 519/1020] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:43.851179 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 502/1021] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:43 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 08:28:43.852125 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84602de-cb52-4c88-bb75-f8e6645ffd07 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:43.864039 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b8cac-07c7-4d68-b4cc-d8be7fcb46f3 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:43.889215 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:43.925750 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:43.961675 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84602de-cb52-4c88-bb75-f8e6645ffd07 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:43.971240 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b8cac-07c7-4d68-b4cc-d8be7fcb46f3 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:43.999131 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f9b60c-7576-432f-8cb4-7c83736e397f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:43.999792 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f9b60c-7576-432f-8cb4-7c83736e397f None admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:44.000080 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f9b60c-7576-432f-8cb4-7c83736e397f None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:44.000359 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f9b60c-7576-432f-8cb4-7c83736e397f None admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:44.001066 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f9b60c-7576-432f-8cb4-7c83736e397f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:44.002553 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f9b60c-7576-432f-8cb4-7c83736e397f None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:44.006330 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7c7fa0-9abc-4faf-9f6c-e3558cbce552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:44.007119 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c7fa0-9abc-4faf-9f6c-e3558cbce552 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:44.007402 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c7fa0-9abc-4faf-9f6c-e3558cbce552 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:44.007673 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c7fa0-9abc-4faf-9f6c-e3558cbce552 None admin] PATH_INFO: `/v3/services/image` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:44.008456 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7c7fa0-9abc-4faf-9f6c-e3558cbce552 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:44.010094 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7c7fa0-9abc-4faf-9f6c-e3558cbce552 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:44.022089 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-2a7c7fa0-9abc-4faf-9f6c-e3558cbce552 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 08:28:44.022089 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:44.022089 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 08:28:44.022089 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 08:28:44.022089 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 08:28:44.022089 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 08:28:44.022089 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 08:28:44.022089 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:28:44.022089 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 08:28:44.022089 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:44.022089 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:28:44.022089 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:44.022089 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:44.022089 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:44.022089 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:44.022089 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:44.022089 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:44.023212 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:44.023212 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:44.023212 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:44.023212 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:44.023212 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:44.023212 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:44.023212 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 08:28:44.023212 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 08:28:44.023212 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 08:28:44.023212 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 08:28:44.023212 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:44.023212 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:44.023212 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:44.023212 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:44.023212 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:44.023212 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:44.025012 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:44.025012 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:44.025012 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:44.025012 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:44.025012 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:44.025012 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:44.025012 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:44.025012 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:44.025012 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:44.025012 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:44.025012 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 08:28:44.025012 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:28:44.025012 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 08:28:44.025012 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:44.025012 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 503/1022] 10.222.0.13 () {56 vars in 1105 bytes} [Fri Jun 24 08:28:43 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:28:44.029828 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7c7fa0-9abc-4faf-9f6c-e3558cbce552 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:44.043695 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 520/1023] 10.222.0.13 () {60 vars in 1141 bytes} [Fri Jun 24 08:28:43 2022] POST /identity/v3/projects => generated 311 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:28:44.138057 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:44.152519 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7c7fa0-9abc-4faf-9f6c-e3558cbce552 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:44.168848 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4edc018-7a92-4ee6-9b9b-861ee63bab87 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:44.169179 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4edc018-7a92-4ee6-9b9b-861ee63bab87 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:44.169374 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4edc018-7a92-4ee6-9b9b-861ee63bab87 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:44.171120 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 521/1024] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:28:44.176458 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4099fb-1b66-41f6-8f4f-0c3805da2d08 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:44.176741 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4099fb-1b66-41f6-8f4f-0c3805da2d08 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:44.176947 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4099fb-1b66-41f6-8f4f-0c3805da2d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:44.191360 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dcb1a8-d324-4396-a9c0-bcac52c0e247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:44.191995 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dcb1a8-d324-4396-a9c0-bcac52c0e247 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:44.192278 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dcb1a8-d324-4396-a9c0-bcac52c0e247 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:44.192554 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dcb1a8-d324-4396-a9c0-bcac52c0e247 None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:44.193263 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dcb1a8-d324-4396-a9c0-bcac52c0e247 None admin] RBAC: Adding query filter params (name=image) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:44.193568 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dcb1a8-d324-4396-a9c0-bcac52c0e247 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:44.194977 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dcb1a8-d324-4396-a9c0-bcac52c0e247 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:44.197781 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-7d4099fb-1b66-41f6-8f4f-0c3805da2d08 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:44.198998 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-7d4099fb-1b66-41f6-8f4f-0c3805da2d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:44.201294 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 504/1025] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:44 2022] GET /identity/v3/services?name=image => generated 122 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:44.207268 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dcb1a8-d324-4396-a9c0-bcac52c0e247 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:44.282138 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 522/1026] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:44 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 08:28:44.306916 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4099fb-1b66-41f6-8f4f-0c3805da2d08 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:44.315768 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dcb1a8-d324-4396-a9c0-bcac52c0e247 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:44.355232 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fc6893-53ab-423c-8baf-f4434b177cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:44.356062 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc6893-53ab-423c-8baf-f4434b177cfb None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:44.356766 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc6893-53ab-423c-8baf-f4434b177cfb None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:44.357054 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc6893-53ab-423c-8baf-f4434b177cfb None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:44.358090 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fc6893-53ab-423c-8baf-f4434b177cfb None admin] RBAC: Adding query filter params (type=image) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:44.358403 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fc6893-53ab-423c-8baf-f4434b177cfb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:44.359799 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:44.360258 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fc6893-53ab-423c-8baf-f4434b177cfb None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:44.367125 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 505/1027] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:44 2022] GET /identity/v3/services?type=image => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:44.373640 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fc6893-53ab-423c-8baf-f4434b177cfb None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:44.394312 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:44.427454 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4099fb-1b66-41f6-8f4f-0c3805da2d08 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:44.465202 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a2ec31-412a-413c-b48b-180acd94c8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:44.466040 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a2ec31-412a-413c-b48b-180acd94c8af None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:44.466329 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a2ec31-412a-413c-b48b-180acd94c8af None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:44.466607 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a2ec31-412a-413c-b48b-180acd94c8af None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:44.477291 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a2ec31-412a-413c-b48b-180acd94c8af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:44.485937 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-92a2ec31-412a-413c-b48b-180acd94c8af None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 08:28:44.485937 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:44.485937 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:44.485937 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:44.485937 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:44.485937 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:44.485937 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:44.485937 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:44.485937 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:44.485937 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:44.485937 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:44.485937 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:44.485937 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:44.485937 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:44.485937 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:44.485937 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:44.487095 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:44.487095 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:44.487095 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:44.487095 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:44.487095 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:44.487095 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:44.487095 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:44.487095 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:44.487095 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:44.487095 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:44.487095 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:44.487095 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:44.487095 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:44.487095 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:44.487095 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:44.487095 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:44.488146 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:44.488146 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:44.488146 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:44.488146 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:44.488146 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:44.488146 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:44.488146 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:44.488146 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:44.488146 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:44.488146 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:44.488146 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 08:28:44.488146 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:44.488146 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 523/1028] 10.222.0.13 () {56 vars in 1120 bytes} [Fri Jun 24 08:28:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:28:44.493050 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a2ec31-412a-413c-b48b-180acd94c8af None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:44.497941 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fc6893-53ab-423c-8baf-f4434b177cfb None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:44.536469 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54866baa-a314-4b66-9a93-f917f6a8ac49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:44.537375 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54866baa-a314-4b66-9a93-f917f6a8ac49 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:44.537862 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54866baa-a314-4b66-9a93-f917f6a8ac49 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:44.538283 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54866baa-a314-4b66-9a93-f917f6a8ac49 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:44.539242 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54866baa-a314-4b66-9a93-f917f6a8ac49 None admin] RBAC: Adding query filter params (interface=public, service_id=baca8d552e25461dbf5cf0e550ed8d89, region_id=RegionOne) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:44.539756 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54866baa-a314-4b66-9a93-f917f6a8ac49 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:44.541427 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54866baa-a314-4b66-9a93-f917f6a8ac49 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:44.549585 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 506/1029] 10.222.0.13 () {56 vars in 1251 bytes} [Fri Jun 24 08:28:44 2022] GET /identity/v3/endpoints?service_id=baca8d552e25461dbf5cf0e550ed8d89&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:44.554816 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54866baa-a314-4b66-9a93-f917f6a8ac49 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:44.608710 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a2ec31-412a-413c-b48b-180acd94c8af None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:44.620174 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:44.645477 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:44.647235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3795f48-d061-4c18-85e3-ab5d56447681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:44.647874 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3795f48-d061-4c18-85e3-ab5d56447681 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:44.648152 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3795f48-d061-4c18-85e3-ab5d56447681 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:44.648442 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3795f48-d061-4c18-85e3-ab5d56447681 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:44.649516 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3795f48-d061-4c18-85e3-ab5d56447681 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:44.649516 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3795f48-d061-4c18-85e3-ab5d56447681 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:44.651078 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3795f48-d061-4c18-85e3-ab5d56447681 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:44.660693 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 524/1030] 10.222.0.13 () {56 vars in 1115 bytes} [Fri Jun 24 08:28:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:44.666153 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3795f48-d061-4c18-85e3-ab5d56447681 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:44.672109 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54866baa-a314-4b66-9a93-f917f6a8ac49 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:44.707422 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a186919-9dcb-429a-a9e0-68dbb98ab326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:44.708009 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a186919-9dcb-429a-a9e0-68dbb98ab326 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:44.708297 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a186919-9dcb-429a-a9e0-68dbb98ab326 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:44.708568 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a186919-9dcb-429a-a9e0-68dbb98ab326 None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:44.709242 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a186919-9dcb-429a-a9e0-68dbb98ab326 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:44.710661 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a186919-9dcb-429a-a9e0-68dbb98ab326 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:44.725727 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 507/1031] 10.222.0.13 () {56 vars in 1087 bytes} [Fri Jun 24 08:28:44 2022] GET /identity/v3/services => generated 1957 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:28:44.787751 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3795f48-d061-4c18-85e3-ab5d56447681 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:44.826427 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cf65fc-469b-4038-b236-a08ca551367a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:44.827244 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf65fc-469b-4038-b236-a08ca551367a None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:44.827531 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf65fc-469b-4038-b236-a08ca551367a None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:44.827806 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf65fc-469b-4038-b236-a08ca551367a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:44.840401 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cf65fc-469b-4038-b236-a08ca551367a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:44.841909 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cf65fc-469b-4038-b236-a08ca551367a None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:44.860967 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-44cf65fc-469b-4038-b236-a08ca551367a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:44.860967 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:44.860967 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:44.860967 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:44.860967 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:44.860967 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:44.860967 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:44.860967 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:44.860967 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:44.860967 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:44.860967 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:44.860967 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:44.860967 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:28:44.860967 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:28:44.860967 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:28:44.862054 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:28:44.862054 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:44.862054 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:44.862054 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:44.862054 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:44.862054 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:44.862054 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:44.862054 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:44.862054 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:44.862054 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:44.862054 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:44.862054 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:44.862054 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:44.862054 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:44.862054 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:44.862054 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:44.862054 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:44.862971 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:28:44.862971 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:28:44.862971 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:44.862971 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:44.862971 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 525/1032] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:28:44.870094 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a186919-9dcb-429a-a9e0-68dbb98ab326 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:44.976967 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a186919-9dcb-429a-a9e0-68dbb98ab326 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:45.012115 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b05e6-4725-48b8-ae2c-ae4441252f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:45.012786 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b05e6-4725-48b8-ae2c-ae4441252f53 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:45.013074 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b05e6-4725-48b8-ae2c-ae4441252f53 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:45.013354 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b05e6-4725-48b8-ae2c-ae4441252f53 None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:45.014102 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480b05e6-4725-48b8-ae2c-ae4441252f53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:45.014413 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480b05e6-4725-48b8-ae2c-ae4441252f53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:45.015793 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480b05e6-4725-48b8-ae2c-ae4441252f53 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:45.028234 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 508/1033] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:28:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:45.033990 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cf65fc-469b-4038-b236-a08ca551367a None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:45.064559 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:45.146041 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cf65fc-469b-4038-b236-a08ca551367a None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:45.182591 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea2dea-a603-4778-a39e-21fb3c5210fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:45.183388 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea2dea-a603-4778-a39e-21fb3c5210fb None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:45.183673 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea2dea-a603-4778-a39e-21fb3c5210fb None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:45.183948 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea2dea-a603-4778-a39e-21fb3c5210fb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:45.192930 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ea2dea-a603-4778-a39e-21fb3c5210fb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:45.195770 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ea2dea-a603-4778-a39e-21fb3c5210fb None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:45.215664 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-24ea2dea-a603-4778-a39e-21fb3c5210fb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 08:28:45.215664 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:45.215664 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:45.215664 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:45.215664 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:45.215664 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:45.215664 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:45.215664 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:45.215664 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:45.215664 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:45.215664 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:45.215664 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:45.215664 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:28:45.215664 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:28:45.215664 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:28:45.215664 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:28:45.216685 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:45.216685 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:45.216685 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:28:45.216685 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:28:45.216685 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:28:45.216685 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:28:45.216685 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:45.216685 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:45.216685 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:45.216685 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:45.216685 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:45.216685 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:45.216685 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:45.216685 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:45.216685 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:45.216685 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:45.216685 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:45.216685 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:45.217635 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:45.217635 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:45.217635 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:28:45.217635 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:28:45.217635 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:28:45.217635 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:28:45.217635 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:28:45.217635 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:28:45.217635 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 08:28:45.217635 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:45.221664 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 526/1034] 10.222.0.13 () {56 vars in 1093 bytes} [Fri Jun 24 08:28:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:28:45.222808 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b05e6-4725-48b8-ae2c-ae4441252f53 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:45.226751 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:45.329778 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b05e6-4725-48b8-ae2c-ae4441252f53 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:45.368382 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4328d2d-77f9-4516-835e-3df6c337361b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:45.369437 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4328d2d-77f9-4516-835e-3df6c337361b None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:45.369962 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4328d2d-77f9-4516-835e-3df6c337361b None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:45.370441 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4328d2d-77f9-4516-835e-3df6c337361b None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:45.371469 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4328d2d-77f9-4516-835e-3df6c337361b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:45.371971 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4328d2d-77f9-4516-835e-3df6c337361b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:45.373727 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4328d2d-77f9-4516-835e-3df6c337361b None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:45.397716 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 509/1035] 10.222.0.13 () {56 vars in 1133 bytes} [Fri Jun 24 08:28:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:45.398795 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea2dea-a603-4778-a39e-21fb3c5210fb None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:45.488279 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:45.505451 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea2dea-a603-4778-a39e-21fb3c5210fb None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:45.541399 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6989b69c-5579-4336-9393-3df4dafba732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:45.542116 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6989b69c-5579-4336-9393-3df4dafba732 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:45.542408 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6989b69c-5579-4336-9393-3df4dafba732 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:45.542687 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6989b69c-5579-4336-9393-3df4dafba732 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:45.610503 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6989b69c-5579-4336-9393-3df4dafba732 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:45.611864 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6989b69c-5579-4336-9393-3df4dafba732 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:45.697728 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:45.720761 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 527/1036] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:28:45 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 08:28:45.722234 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4328d2d-77f9-4516-835e-3df6c337361b None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:45.786015 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:45.826932 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4328d2d-77f9-4516-835e-3df6c337361b None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:45.865343 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b559fad0-2a0f-48fd-87a6-af8ee6fc916a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:45.866088 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559fad0-2a0f-48fd-87a6-af8ee6fc916a None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:45.866375 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559fad0-2a0f-48fd-87a6-af8ee6fc916a None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:45.866652 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559fad0-2a0f-48fd-87a6-af8ee6fc916a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:45.877625 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b559fad0-2a0f-48fd-87a6-af8ee6fc916a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:45.879471 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b559fad0-2a0f-48fd-87a6-af8ee6fc916a None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:45.902650 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-b559fad0-2a0f-48fd-87a6-af8ee6fc916a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:45.902650 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:45.902650 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:45.902650 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:45.902650 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:45.902650 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:45.902650 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:45.902650 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:45.902650 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:45.902650 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:45.902650 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:45.902650 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:45.902650 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:28:45.902650 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:28:45.902650 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:28:45.904154 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:28:45.904154 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:45.904154 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:45.904154 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:45.904154 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:45.904154 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:45.904154 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:45.904154 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:45.904154 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:45.904154 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:45.904154 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:45.904154 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:45.904154 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:45.904154 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:45.904154 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:45.904154 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:45.904154 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:45.905827 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:28:45.905827 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:28:45.905827 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:45.905827 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:45.905827 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 510/1037] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:28:45.909470 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6989b69c-5579-4336-9393-3df4dafba732 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:46.010826 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6989b69c-5579-4336-9393-3df4dafba732 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:46.045945 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f00cddb-9760-4284-8dc5-d679a11af7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:46.046605 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f00cddb-9760-4284-8dc5-d679a11af7df None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:46.046888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f00cddb-9760-4284-8dc5-d679a11af7df None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:46.047163 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f00cddb-9760-4284-8dc5-d679a11af7df None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:46.047876 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f00cddb-9760-4284-8dc5-d679a11af7df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:46.048176 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f00cddb-9760-4284-8dc5-d679a11af7df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:46.049526 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f00cddb-9760-4284-8dc5-d679a11af7df None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:46.061618 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 528/1038] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:28:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:46.067011 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b559fad0-2a0f-48fd-87a6-af8ee6fc916a None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:46.139595 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24da764-c762-43e3-9d8d-a601f633cd84 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:46.139894 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24da764-c762-43e3-9d8d-a601f633cd84 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:46.140079 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24da764-c762-43e3-9d8d-a601f633cd84 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:46.141432 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 529/1039] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:28:46.146305 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff24527-fd5e-45db-b842-dec7c768b556 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:46.146660 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff24527-fd5e-45db-b842-dec7c768b556 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:46.146944 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff24527-fd5e-45db-b842-dec7c768b556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:46.163231 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-1ff24527-fd5e-45db-b842-dec7c768b556 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:46.164398 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-1ff24527-fd5e-45db-b842-dec7c768b556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:46.183825 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b559fad0-2a0f-48fd-87a6-af8ee6fc916a None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:46.199190 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:46.206301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:46.222290 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3009e3-5e8a-40fd-b436-88bef3ed4029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:46.223173 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3009e3-5e8a-40fd-b436-88bef3ed4029 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:46.223465 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3009e3-5e8a-40fd-b436-88bef3ed4029 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:46.223742 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3009e3-5e8a-40fd-b436-88bef3ed4029 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:46.230621 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3009e3-5e8a-40fd-b436-88bef3ed4029 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:46.232076 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3009e3-5e8a-40fd-b436-88bef3ed4029 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:46.235102 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 530/1040] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:46 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 08:28:46.246656 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-eb3009e3-5e8a-40fd-b436-88bef3ed4029 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:28:46.246656 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:46.246656 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:46.246656 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:46.246656 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:46.246656 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:46.246656 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:46.246656 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:46.246656 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:46.246656 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:46.246656 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:46.246656 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:46.246656 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:28:46.246656 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:28:46.246656 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:28:46.247532 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:28:46.247532 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:46.247532 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:46.247532 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:46.247532 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:46.247532 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:46.247532 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:46.247532 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:46.247532 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:46.247532 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:46.247532 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:46.247532 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:46.247532 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:46.247532 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:46.247532 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:46.247532 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:46.247532 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:46.248413 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:28:46.248413 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:28:46.248413 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:28:46.248413 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:28:46.248413 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:28:46.248413 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:28:46.248413 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:28:46.248413 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:46.248413 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 511/1041] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:28:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:46.257335 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff24527-fd5e-45db-b842-dec7c768b556 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:46.262171 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3009e3-5e8a-40fd-b436-88bef3ed4029 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:46.363889 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff24527-fd5e-45db-b842-dec7c768b556 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:46.386575 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3009e3-5e8a-40fd-b436-88bef3ed4029 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:46.402479 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0ff800-9af6-475e-b03b-7d97e2bdd175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:46.403167 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0ff800-9af6-475e-b03b-7d97e2bdd175 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:46.403455 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0ff800-9af6-475e-b03b-7d97e2bdd175 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:46.403739 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0ff800-9af6-475e-b03b-7d97e2bdd175 None admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:46.404526 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0ff800-9af6-475e-b03b-7d97e2bdd175 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:46.404843 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0ff800-9af6-475e-b03b-7d97e2bdd175 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:46.406307 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0ff800-9af6-475e-b03b-7d97e2bdd175 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:46.418906 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 531/1042] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Jun 24 08:28:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:46.424633 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0ff800-9af6-475e-b03b-7d97e2bdd175 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:46.427385 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdcdee9-df98-475d-9f00-51f21c10ac08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:46.428676 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdcdee9-df98-475d-9f00-51f21c10ac08 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:46.428676 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdcdee9-df98-475d-9f00-51f21c10ac08 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:46.428676 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdcdee9-df98-475d-9f00-51f21c10ac08 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:46.429887 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdcdee9-df98-475d-9f00-51f21c10ac08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:46.432016 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdcdee9-df98-475d-9f00-51f21c10ac08 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:46.433857 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 512/1043] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:46.439452 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdcdee9-df98-475d-9f00-51f21c10ac08 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:46.460622 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:46.494443 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:46.539531 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0ff800-9af6-475e-b03b-7d97e2bdd175 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:46.557139 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdcdee9-df98-475d-9f00-51f21c10ac08 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:46.582726 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04dfd5a-941a-407b-9947-10854f758811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:46.583710 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04dfd5a-941a-407b-9947-10854f758811 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:46.584155 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04dfd5a-941a-407b-9947-10854f758811 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:46.584437 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04dfd5a-941a-407b-9947-10854f758811 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:46.585267 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04dfd5a-941a-407b-9947-10854f758811 None admin] RBAC: Adding query filter params (role.id=f9658866044e4fdeaddcc91781166faf, scope.project.id=180b0af5863349e99ae412b07c1e9465, user.id=74bb1bc6821b4500b26f160ccae5b7d5) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:46.585582 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04dfd5a-941a-407b-9947-10854f758811 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:46.587994 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04dfd5a-941a-407b-9947-10854f758811 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:46.594092 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 532/1044] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Jun 24 08:28:46 2022] GET /identity/v3/role_assignments?user.id=74bb1bc6821b4500b26f160ccae5b7d5&scope.project.id=180b0af5863349e99ae412b07c1e9465&role.id=f9658866044e4fdeaddcc91781166faf => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:46.603597 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8129116c-0c14-4ae0-ad5a-e6cd714ea160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:46.604294 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129116c-0c14-4ae0-ad5a-e6cd714ea160 None admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:46.604582 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129116c-0c14-4ae0-ad5a-e6cd714ea160 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:46.604858 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129116c-0c14-4ae0-ad5a-e6cd714ea160 None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:46.605596 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8129116c-0c14-4ae0-ad5a-e6cd714ea160 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:46.607160 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8129116c-0c14-4ae0-ad5a-e6cd714ea160 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:46.630038 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 513/1045] 10.222.0.13 () {60 vars in 1132 bytes} [Fri Jun 24 08:28:46 2022] POST /identity/v3/users => generated 285 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:28:46.879608 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa9622-12b5-4e69-856e-c84ab30b535d None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:46.880132 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa9622-12b5-4e69-856e-c84ab30b535d None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:46.880429 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa9622-12b5-4e69-856e-c84ab30b535d None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:46.882677 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 533/1046] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:28:46.889850 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027fefb7-48a0-44bb-bb9f-f91c12a234d3 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:46.890490 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027fefb7-48a0-44bb-bb9f-f91c12a234d3 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:46.890490 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027fefb7-48a0-44bb-bb9f-f91c12a234d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:46.909123 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-027fefb7-48a0-44bb-bb9f-f91c12a234d3 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:46.910747 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-027fefb7-48a0-44bb-bb9f-f91c12a234d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:46.989010 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 514/1047] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:46 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:28:47.013549 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aa9622-12b5-4e69-856e-c84ab30b535d None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:47.114942 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:47.120243 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aa9622-12b5-4e69-856e-c84ab30b535d None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:47.155493 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05ff29-6a8f-4de1-9613-555c4956fc2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:47.156198 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05ff29-6a8f-4de1-9613-555c4956fc2b None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:47.156482 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05ff29-6a8f-4de1-9613-555c4956fc2b None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:47.156758 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05ff29-6a8f-4de1-9613-555c4956fc2b None admin] PATH_INFO: `/v3/services/image` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:47.157404 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac05ff29-6a8f-4de1-9613-555c4956fc2b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:47.158800 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac05ff29-6a8f-4de1-9613-555c4956fc2b None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:47.170266 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-ac05ff29-6a8f-4de1-9613-555c4956fc2b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 08:28:47.170266 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:47.170266 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 08:28:47.170266 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 08:28:47.170266 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 08:28:47.170266 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 08:28:47.170266 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 08:28:47.170266 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:28:47.170266 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 08:28:47.170266 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:47.170266 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:28:47.170266 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:47.170266 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:47.170266 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:47.170266 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:47.170266 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:47.170266 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:47.171252 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:47.171252 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:47.171252 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:47.171252 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:47.171252 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:47.171252 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:47.171252 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 08:28:47.171252 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 08:28:47.171252 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 08:28:47.171252 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 08:28:47.171252 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:47.171252 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:47.171252 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:47.171252 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:47.171252 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:47.171252 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:47.172091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:47.172091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:47.172091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:47.172091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:47.172091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:47.172091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:47.172091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:47.172091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:47.172091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:47.172091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:47.172091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 08:28:47.172091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:28:47.172091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 08:28:47.172091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:47.173783 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 534/1048] 10.222.0.13 () {56 vars in 1105 bytes} [Fri Jun 24 08:28:47 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:47.177662 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027fefb7-48a0-44bb-bb9f-f91c12a234d3 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:47.208524 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:47.295118 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027fefb7-48a0-44bb-bb9f-f91c12a234d3 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:47.329948 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae7e34-6e49-49a7-ac86-e5f06a9a7687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:47.330608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae7e34-6e49-49a7-ac86-e5f06a9a7687 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:47.330891 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae7e34-6e49-49a7-ac86-e5f06a9a7687 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:47.331166 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae7e34-6e49-49a7-ac86-e5f06a9a7687 None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:47.331865 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae7e34-6e49-49a7-ac86-e5f06a9a7687 None admin] RBAC: Adding query filter params (name=image) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:47.332173 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae7e34-6e49-49a7-ac86-e5f06a9a7687 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:47.333542 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae7e34-6e49-49a7-ac86-e5f06a9a7687 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:47.343643 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 515/1049] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:47 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:47.344495 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05ff29-6a8f-4de1-9613-555c4956fc2b None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:47.451638 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05ff29-6a8f-4de1-9613-555c4956fc2b None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:47.489869 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74b80db-814b-459b-9855-bed91345dbc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:47.490529 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74b80db-814b-459b-9855-bed91345dbc5 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:47.490815 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74b80db-814b-459b-9855-bed91345dbc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:47.491084 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74b80db-814b-459b-9855-bed91345dbc5 None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:47.491782 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74b80db-814b-459b-9855-bed91345dbc5 None admin] RBAC: Adding query filter params (type=image) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:47.492140 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74b80db-814b-459b-9855-bed91345dbc5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:47.494213 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74b80db-814b-459b-9855-bed91345dbc5 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:47.504027 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 535/1050] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:47 2022] GET /identity/v3/services?type=image => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:47.505131 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae7e34-6e49-49a7-ac86-e5f06a9a7687 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:47.597725 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:47.608971 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae7e34-6e49-49a7-ac86-e5f06a9a7687 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:47.643238 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa26675-f0e8-4aed-907a-ccdc29a6e752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:47.643808 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa26675-f0e8-4aed-907a-ccdc29a6e752 None admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:47.644092 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa26675-f0e8-4aed-907a-ccdc29a6e752 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:47.644367 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa26675-f0e8-4aed-907a-ccdc29a6e752 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:47.645112 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa26675-f0e8-4aed-907a-ccdc29a6e752 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:47.646490 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa26675-f0e8-4aed-907a-ccdc29a6e752 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:47.670735 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 516/1051] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Jun 24 08:28:47 2022] POST /identity/v3/endpoints => generated 327 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:28:48.640278 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3eea1-2f98-47a5-af41-e4a646e1b97f None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:48.640773 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3eea1-2f98-47a5-af41-e4a646e1b97f None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:48.640773 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3eea1-2f98-47a5-af41-e4a646e1b97f None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:48.642278 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 536/1052] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:48.647207 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ecd00-f073-4925-a346-8bcd5770db24 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:48.647605 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ecd00-f073-4925-a346-8bcd5770db24 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:48.647781 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ecd00-f073-4925-a346-8bcd5770db24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:48.666396 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-c13ecd00-f073-4925-a346-8bcd5770db24 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:48.667567 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-c13ecd00-f073-4925-a346-8bcd5770db24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:48.742334 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 517/1053] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:28:48.765456 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a3eea1-2f98-47a5-af41-e4a646e1b97f None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:48.785841 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff02ec78-c547-4315-b32f-115f92144fa3 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:48.786137 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff02ec78-c547-4315-b32f-115f92144fa3 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:48.786421 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff02ec78-c547-4315-b32f-115f92144fa3 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:48.787930 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 518/1054] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:28:48.792698 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece38c2-db24-45f5-876e-623a59f3c203 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:48.792988 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece38c2-db24-45f5-876e-623a59f3c203 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:48.793272 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece38c2-db24-45f5-876e-623a59f3c203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:48.794990 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:48.811698 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-9ece38c2-db24-45f5-876e-623a59f3c203 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:48.813003 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-9ece38c2-db24-45f5-876e-623a59f3c203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:48.877443 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a3eea1-2f98-47a5-af41-e4a646e1b97f None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:48.893288 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 519/1055] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 08:28:48.917340 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef24f8bb-0e97-4707-b85e-ed1ca9a74363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:48.917572 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ece38c2-db24-45f5-876e-623a59f3c203 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:48.918323 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef24f8bb-0e97-4707-b85e-ed1ca9a74363 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:48.918610 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef24f8bb-0e97-4707-b85e-ed1ca9a74363 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:48.918887 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef24f8bb-0e97-4707-b85e-ed1ca9a74363 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:48.920483 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:48.933017 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef24f8bb-0e97-4707-b85e-ed1ca9a74363 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:48.941118 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-ef24f8bb-0e97-4707-b85e-ed1ca9a74363 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 08:28:48.941118 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:48.941118 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:48.941118 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:48.941118 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:48.941118 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:48.941118 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:48.941118 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:48.941118 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:48.941118 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:48.941118 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:48.941118 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:48.941118 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:48.941118 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:48.941118 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:48.941118 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:48.942098 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:48.942098 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:48.942098 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:48.942098 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:48.942098 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:48.942098 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:48.942098 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:48.942098 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:48.942098 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:48.942098 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:48.942098 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:48.942098 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:48.942098 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:48.942098 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:48.942098 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:48.942098 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:48.942946 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:48.942946 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:48.942946 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:48.942946 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:48.942946 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:48.942946 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:48.942946 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:48.942946 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:48.942946 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:48.942946 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:48.942946 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 08:28:48.942946 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:48.942946 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 537/1056] 10.222.0.13 () {56 vars in 1120 bytes} [Fri Jun 24 08:28:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:48.949513 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef24f8bb-0e97-4707-b85e-ed1ca9a74363 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:49.028246 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ece38c2-db24-45f5-876e-623a59f3c203 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:49.039300 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:49.060250 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef24f8bb-0e97-4707-b85e-ed1ca9a74363 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:49.065043 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faee03a-859f-4cfe-ae9b-bf9e1197940b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:49.065875 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faee03a-859f-4cfe-ae9b-bf9e1197940b None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:49.066162 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faee03a-859f-4cfe-ae9b-bf9e1197940b None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:49.066435 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faee03a-859f-4cfe-ae9b-bf9e1197940b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:49.077823 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faee03a-859f-4cfe-ae9b-bf9e1197940b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:49.085306 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-8faee03a-859f-4cfe-ae9b-bf9e1197940b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:28:49.085306 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:49.085306 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:49.085306 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:49.085306 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:49.085306 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:49.085306 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:49.085306 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:49.085306 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:49.085306 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:49.085306 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:49.085306 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:49.085306 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:49.085306 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:49.085306 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:49.085306 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:49.086372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:49.086372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:49.086372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:49.086372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:49.086372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:49.086372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:49.086372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:49.086372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:49.086372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:49.086372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:49.086372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:49.086372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:49.086372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:49.086372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:49.086372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:49.086372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:49.088565 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:49.088565 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:49.088565 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:49.088565 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:49.088565 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:49.088565 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:49.088565 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:49.088565 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:49.088565 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:49.088565 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:49.088565 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:28:49.088565 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:49.088565 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 520/1057] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Jun 24 08:28:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:28:49.093182 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faee03a-859f-4cfe-ae9b-bf9e1197940b None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:49.100382 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd4a2d7-ff6f-45e8-bbf0-dab11c63c352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:49.101057 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd4a2d7-ff6f-45e8-bbf0-dab11c63c352 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:49.101343 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd4a2d7-ff6f-45e8-bbf0-dab11c63c352 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:49.101621 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd4a2d7-ff6f-45e8-bbf0-dab11c63c352 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:49.103041 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd4a2d7-ff6f-45e8-bbf0-dab11c63c352 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:49.103357 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd4a2d7-ff6f-45e8-bbf0-dab11c63c352 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:49.104743 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd4a2d7-ff6f-45e8-bbf0-dab11c63c352 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:49.113053 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 538/1058] 10.222.0.13 () {56 vars in 1115 bytes} [Fri Jun 24 08:28:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:49.119706 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd4a2d7-ff6f-45e8-bbf0-dab11c63c352 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:49.165728 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:49.207496 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faee03a-859f-4cfe-ae9b-bf9e1197940b None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:49.224579 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd4a2d7-ff6f-45e8-bbf0-dab11c63c352 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:49.245732 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60120ccb-4e93-4f82-8778-a71555431893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:49.246733 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60120ccb-4e93-4f82-8778-a71555431893 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:49.247164 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60120ccb-4e93-4f82-8778-a71555431893 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:49.247582 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60120ccb-4e93-4f82-8778-a71555431893 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:49.248492 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60120ccb-4e93-4f82-8778-a71555431893 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:49.249068 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60120ccb-4e93-4f82-8778-a71555431893 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:49.250735 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60120ccb-4e93-4f82-8778-a71555431893 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:49.261187 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 521/1059] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:28:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:49.264415 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1687e60-343b-49fc-bbae-e259b5f43def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:49.265266 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1687e60-343b-49fc-bbae-e259b5f43def None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:49.265577 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1687e60-343b-49fc-bbae-e259b5f43def None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:49.265866 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1687e60-343b-49fc-bbae-e259b5f43def None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:49.266954 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60120ccb-4e93-4f82-8778-a71555431893 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:49.277351 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1687e60-343b-49fc-bbae-e259b5f43def None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:49.278842 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1687e60-343b-49fc-bbae-e259b5f43def None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:49.287085 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:49.298552 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-b1687e60-343b-49fc-bbae-e259b5f43def None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:49.298552 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:49.298552 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:49.298552 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:49.298552 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:49.298552 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:49.298552 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:49.298552 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:49.298552 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:49.298552 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:49.298552 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:49.298552 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:49.298552 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:28:49.298552 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:28:49.298552 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:28:49.299484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:28:49.299484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:49.299484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:49.299484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:49.299484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:49.299484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:49.299484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:49.299484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:49.299484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:49.299484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:49.299484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:49.299484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:49.299484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:49.299484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:49.299484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:49.299484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:49.299484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:49.300375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:28:49.300375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:28:49.300375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:49.300375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:49.300375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 539/1060] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:28:49.305899 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1687e60-343b-49fc-bbae-e259b5f43def None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:49.380454 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60120ccb-4e93-4f82-8778-a71555431893 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:49.407368 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:49.418430 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eef4349-2715-4362-a45c-ea798941e498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:49.419309 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef4349-2715-4362-a45c-ea798941e498 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:49.419595 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef4349-2715-4362-a45c-ea798941e498 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:49.419898 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef4349-2715-4362-a45c-ea798941e498 None admin] PATH_INFO: `/v3/users/4af9c25b61c949da9590c3ee0ea0b1cf` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:49.433158 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1687e60-343b-49fc-bbae-e259b5f43def None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:49.436345 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eef4349-2715-4362-a45c-ea798941e498 None admin] RBAC: Authorizing `identity:get_user(user_id=4af9c25b61c949da9590c3ee0ea0b1cf)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:49.437817 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eef4349-2715-4362-a45c-ea798941e498 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:49.444112 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 522/1061] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Jun 24 08:28:49 2022] GET /identity/v3/users/4af9c25b61c949da9590c3ee0ea0b1cf => generated 285 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:49.444964 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eef4349-2715-4362-a45c-ea798941e498 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:49.474551 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318f4b14-4721-4017-819e-26db9eed55c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:49.475432 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f4b14-4721-4017-819e-26db9eed55c6 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:49.475568 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f4b14-4721-4017-819e-26db9eed55c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:49.475781 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f4b14-4721-4017-819e-26db9eed55c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:49.476498 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318f4b14-4721-4017-819e-26db9eed55c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:49.476839 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318f4b14-4721-4017-819e-26db9eed55c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:49.479102 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318f4b14-4721-4017-819e-26db9eed55c6 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:49.494269 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 540/1062] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:28:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:49.499490 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318f4b14-4721-4017-819e-26db9eed55c6 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:49.553793 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eef4349-2715-4362-a45c-ea798941e498 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:49.568301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:49.593637 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe2e7b2-c9d7-41f8-a2fb-55efe89cd545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:49.594313 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2e7b2-c9d7-41f8-a2fb-55efe89cd545 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:49.594600 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2e7b2-c9d7-41f8-a2fb-55efe89cd545 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:49.594879 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2e7b2-c9d7-41f8-a2fb-55efe89cd545 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:49.619897 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318f4b14-4721-4017-819e-26db9eed55c6 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:49.651016 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:49.663788 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8ee157-ebe9-4a3c-a28c-c06e793594ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:49.664662 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ee157-ebe9-4a3c-a28c-c06e793594ef None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:49.664974 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ee157-ebe9-4a3c-a28c-c06e793594ef None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:49.665490 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ee157-ebe9-4a3c-a28c-c06e793594ef None admin] PATH_INFO: `/v3/users/nova` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:49.668807 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe2e7b2-c9d7-41f8-a2fb-55efe89cd545 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:49.670232 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe2e7b2-c9d7-41f8-a2fb-55efe89cd545 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:49.675323 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8ee157-ebe9-4a3c-a28c-c06e793594ef None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:49.676761 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8ee157-ebe9-4a3c-a28c-c06e793594ef None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:49.693009 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-fe8ee157-ebe9-4a3c-a28c-c06e793594ef None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 08:28:49.693009 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:49.693009 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:49.693009 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:49.693009 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:49.693009 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:49.693009 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:49.693009 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:49.693009 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:49.693009 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:49.693009 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:49.693009 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:49.693009 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:28:49.693009 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:28:49.693009 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:28:49.693009 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:28:49.694048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:49.694048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:49.694048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:28:49.694048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:28:49.694048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:28:49.694048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:28:49.694048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:49.694048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:49.694048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:49.694048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:49.694048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:49.694048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:49.694048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:49.694048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:49.694048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:49.694048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:49.694048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:49.694048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:49.695733 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:49.695733 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:49.695733 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:28:49.695733 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:28:49.695733 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:28:49.695733 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:28:49.695733 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:28:49.695733 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:28:49.695733 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 08:28:49.695733 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:49.695733 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 541/1063] 10.222.0.13 () {56 vars in 1093 bytes} [Fri Jun 24 08:28:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:28:49.700342 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8ee157-ebe9-4a3c-a28c-c06e793594ef None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:49.782126 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 523/1064] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:28:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 24 08:28:49.787820 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d4b1f-513e-4720-b2b6-a447a710f7c7 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:49.788112 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d4b1f-513e-4720-b2b6-a447a710f7c7 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:49.788387 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d4b1f-513e-4720-b2b6-a447a710f7c7 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:49.790284 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 524/1065] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:28:49.790998 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8d4b1f-513e-4720-b2b6-a447a710f7c7 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:49.818046 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8ee157-ebe9-4a3c-a28c-c06e793594ef None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:49.838236 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:49.854846 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b92f814-8739-40db-9277-58c20338d313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:49.855517 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92f814-8739-40db-9277-58c20338d313 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:49.855803 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92f814-8739-40db-9277-58c20338d313 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:49.856080 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92f814-8739-40db-9277-58c20338d313 None admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:49.856918 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b92f814-8739-40db-9277-58c20338d313 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:49.857231 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b92f814-8739-40db-9277-58c20338d313 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:49.858693 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b92f814-8739-40db-9277-58c20338d313 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:49.875894 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 542/1066] 10.222.0.13 () {56 vars in 1133 bytes} [Fri Jun 24 08:28:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:49.877118 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5bca1-09aa-45c8-87e6-c4ba664e35db None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:49.877405 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5bca1-09aa-45c8-87e6-c4ba664e35db None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:49.877684 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5bca1-09aa-45c8-87e6-c4ba664e35db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:49.882968 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:49.898121 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8d4b1f-513e-4720-b2b6-a447a710f7c7 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:49.902801 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-97e5bca1-09aa-45c8-87e6-c4ba664e35db None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:49.904162 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-97e5bca1-09aa-45c8-87e6-c4ba664e35db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:49.942722 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d4c106-d579-487c-9c7c-19b0b11fbb45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:49.943499 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d4c106-d579-487c-9c7c-19b0b11fbb45 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:49.943924 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d4c106-d579-487c-9c7c-19b0b11fbb45 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:49.944099 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d4c106-d579-487c-9c7c-19b0b11fbb45 None admin] PATH_INFO: `/v3/projects/3997d3abea6b443b9df2e09c7ad2092e` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:49.945279 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d4c106-d579-487c-9c7c-19b0b11fbb45 None admin] RBAC: Authorizing `identity:get_project(project_id=3997d3abea6b443b9df2e09c7ad2092e)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:49.947224 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d4c106-d579-487c-9c7c-19b0b11fbb45 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:49.949576 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 525/1067] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:28:49 2022] GET /identity/v3/projects/3997d3abea6b443b9df2e09c7ad2092e => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:49.950366 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d4c106-d579-487c-9c7c-19b0b11fbb45 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:49.990540 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 543/1068] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:28:49.991366 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e5bca1-09aa-45c8-87e6-c4ba664e35db None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:50.056288 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d4c106-d579-487c-9c7c-19b0b11fbb45 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:50.096100 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90543590-931a-4951-91a5-2921df194593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:50.096924 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90543590-931a-4951-91a5-2921df194593 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:50.097208 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90543590-931a-4951-91a5-2921df194593 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:50.097484 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90543590-931a-4951-91a5-2921df194593 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:50.098529 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e5bca1-09aa-45c8-87e6-c4ba664e35db None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:50.112343 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:50.112838 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90543590-931a-4951-91a5-2921df194593 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:50.114356 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90543590-931a-4951-91a5-2921df194593 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:50.137609 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-90543590-931a-4951-91a5-2921df194593 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:50.137609 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:50.137609 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:50.137609 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:50.137609 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:50.137609 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:50.137609 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:50.137609 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:50.137609 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:50.137609 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:50.137609 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:50.137609 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:50.137609 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:28:50.137609 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:28:50.137609 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:28:50.138622 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:28:50.138622 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:50.138622 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:50.138622 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:50.138622 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:50.138622 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:50.138622 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:50.138622 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:50.138622 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:50.138622 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:50.138622 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:50.138622 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:50.138622 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:50.138622 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:50.138622 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:50.138622 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:50.138622 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:50.139544 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:28:50.139544 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:28:50.139544 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:50.139544 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:50.140421 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d7ace0-62de-46fd-b668-70445ab82f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:50.141079 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7ace0-62de-46fd-b668-70445ab82f0b None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:50.141362 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7ace0-62de-46fd-b668-70445ab82f0b None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:50.141635 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7ace0-62de-46fd-b668-70445ab82f0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:50.142491 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d7ace0-62de-46fd-b668-70445ab82f0b None admin] RBAC: Adding query filter params (role.id=acfe707eee6944ad94d1915fc9ee7ea3, scope.project.id=3997d3abea6b443b9df2e09c7ad2092e, user.id=4af9c25b61c949da9590c3ee0ea0b1cf) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:50.142798 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d7ace0-62de-46fd-b668-70445ab82f0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:50.144470 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d7ace0-62de-46fd-b668-70445ab82f0b None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:50.144695 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 526/1069] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:50.146031 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90543590-931a-4951-91a5-2921df194593 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:50.151233 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:50.165734 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 544/1070] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Jun 24 08:28:49 2022] GET /identity/v3/role_assignments?user.id=4af9c25b61c949da9590c3ee0ea0b1cf&scope.project.id=3997d3abea6b443b9df2e09c7ad2092e&role.id=acfe707eee6944ad94d1915fc9ee7ea3 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:50.165734 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d7ace0-62de-46fd-b668-70445ab82f0b None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:50.268598 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90543590-931a-4951-91a5-2921df194593 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:50.299285 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d7ace0-62de-46fd-b668-70445ab82f0b None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:50.317287 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a47878-1ae5-4cab-a326-2d30b57441b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:50.318595 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a47878-1ae5-4cab-a326-2d30b57441b9 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:50.319081 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a47878-1ae5-4cab-a326-2d30b57441b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:50.319816 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a47878-1ae5-4cab-a326-2d30b57441b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:50.342977 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9de9034-cb50-4bc5-ae99-7d9a4bf2a192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:50.343626 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de9034-cb50-4bc5-ae99-7d9a4bf2a192 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:50.343914 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de9034-cb50-4bc5-ae99-7d9a4bf2a192 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:50.344187 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de9034-cb50-4bc5-ae99-7d9a4bf2a192 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:50.344912 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9de9034-cb50-4bc5-ae99-7d9a4bf2a192 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:50.345346 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9de9034-cb50-4bc5-ae99-7d9a4bf2a192 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:50.346637 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9de9034-cb50-4bc5-ae99-7d9a4bf2a192 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:50.363234 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 545/1071] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:28:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:50.368909 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9de9034-cb50-4bc5-ae99-7d9a4bf2a192 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:50.407712 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:50.416646 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a47878-1ae5-4cab-a326-2d30b57441b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:50.418232 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a47878-1ae5-4cab-a326-2d30b57441b9 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:50.423092 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:50.496992 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9de9034-cb50-4bc5-ae99-7d9a4bf2a192 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:50.537639 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017b9544-e9af-47b8-b336-e41419efce2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:50.538379 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b9544-e9af-47b8-b336-e41419efce2c None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:50.538661 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b9544-e9af-47b8-b336-e41419efce2c None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:50.538937 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b9544-e9af-47b8-b336-e41419efce2c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:50.543002 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 527/1072] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:28:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 08:28:50.547386 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017b9544-e9af-47b8-b336-e41419efce2c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:50.547687 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017b9544-e9af-47b8-b336-e41419efce2c None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:50.548704 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a47878-1ae5-4cab-a326-2d30b57441b9 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:50.562249 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-017b9544-e9af-47b8-b336-e41419efce2c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:28:50.562249 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:50.562249 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:50.562249 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:50.562249 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:50.562249 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:50.562249 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:50.562249 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:50.562249 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:50.562249 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:50.562249 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:50.562249 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:50.562249 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:28:50.562249 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:28:50.562249 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:28:50.563265 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:28:50.563265 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:50.563265 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:50.563265 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:50.563265 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:50.563265 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:50.563265 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:50.563265 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:50.563265 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:50.563265 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:50.563265 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:50.563265 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:50.563265 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:50.563265 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:50.563265 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:50.563265 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:50.563265 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:50.564143 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:28:50.564143 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:28:50.564143 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:28:50.564143 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:28:50.564143 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:28:50.564143 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:28:50.564143 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:28:50.564143 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:50.568304 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 546/1073] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:28:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:50.569099 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017b9544-e9af-47b8-b336-e41419efce2c None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:50.676027 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a47878-1ae5-4cab-a326-2d30b57441b9 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:50.678784 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:50.683771 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:50.694827 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017b9544-e9af-47b8-b336-e41419efce2c None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:50.721443 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b207dbd5-10ba-4301-b083-4600c866b1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:50.722264 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b207dbd5-10ba-4301-b083-4600c866b1b5 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:50.722546 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b207dbd5-10ba-4301-b083-4600c866b1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:50.722815 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b207dbd5-10ba-4301-b083-4600c866b1b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:50.736773 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b207dbd5-10ba-4301-b083-4600c866b1b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:50.738281 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b207dbd5-10ba-4301-b083-4600c866b1b5 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:50.740402 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc2ba9f-ec32-4867-b3d3-96d473fd59e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:50.741075 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2ba9f-ec32-4867-b3d3-96d473fd59e6 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:50.741357 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2ba9f-ec32-4867-b3d3-96d473fd59e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:50.741633 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2ba9f-ec32-4867-b3d3-96d473fd59e6 None admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:50.742559 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc2ba9f-ec32-4867-b3d3-96d473fd59e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:50.743033 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc2ba9f-ec32-4867-b3d3-96d473fd59e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:50.744536 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc2ba9f-ec32-4867-b3d3-96d473fd59e6 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:50.762972 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 547/1074] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Jun 24 08:28:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:50.763169 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-b207dbd5-10ba-4301-b083-4600c866b1b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:50.763169 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:50.763169 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:50.763169 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:50.763169 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:50.763169 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:50.763169 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:50.763169 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:50.763169 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:50.763169 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:50.763169 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:50.763169 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:50.763169 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:28:50.763169 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:28:50.763169 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:28:50.764043 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:28:50.764043 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:50.764043 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:50.764043 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:50.764043 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:50.764043 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:50.764043 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:50.764043 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:50.764043 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:50.764043 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:50.764043 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:50.764043 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:50.764043 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:50.764043 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:50.764043 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:50.764043 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:50.764043 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:50.764939 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:28:50.764939 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:28:50.764939 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:50.764939 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:50.764939 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 528/1075] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:50.766431 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc2ba9f-ec32-4867-b3d3-96d473fd59e6 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:50.770102 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b207dbd5-10ba-4301-b083-4600c866b1b5 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:50.894033 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc2ba9f-ec32-4867-b3d3-96d473fd59e6 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:50.900119 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b207dbd5-10ba-4301-b083-4600c866b1b5 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:50.938515 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feed469-58d8-4841-a766-c77778cdbcff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:50.939778 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feed469-58d8-4841-a766-c77778cdbcff None admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:50.940066 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feed469-58d8-4841-a766-c77778cdbcff None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:50.940194 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f3c67-73d5-470c-905c-fe15798f2777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:50.940375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feed469-58d8-4841-a766-c77778cdbcff None admin] PATH_INFO: `/v3/projects/180b0af5863349e99ae412b07c1e9465/users/74bb1bc6821b4500b26f160ccae5b7d5/roles/f9658866044e4fdeaddcc91781166faf` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:50.941628 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f3c67-73d5-470c-905c-fe15798f2777 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:50.941628 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f3c67-73d5-470c-905c-fe15798f2777 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:50.941628 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f3c67-73d5-470c-905c-fe15798f2777 None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:50.942365 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444f3c67-73d5-470c-905c-fe15798f2777 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:50.942759 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444f3c67-73d5-470c-905c-fe15798f2777 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:50.944143 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444f3c67-73d5-470c-905c-fe15798f2777 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:50.948560 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feed469-58d8-4841-a766-c77778cdbcff None admin] RBAC: Authorizing `identity:create_grant(project_id=180b0af5863349e99ae412b07c1e9465, user_id=74bb1bc6821b4500b26f160ccae5b7d5, role_id=f9658866044e4fdeaddcc91781166faf)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:50.950043 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feed469-58d8-4841-a766-c77778cdbcff None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:50.957927 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 529/1076] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:28:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:50.966058 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 548/1077] 10.222.0.13 () {58 vars in 1439 bytes} [Fri Jun 24 08:28:50 2022] PUT /identity/v3/projects/180b0af5863349e99ae412b07c1e9465/users/74bb1bc6821b4500b26f160ccae5b7d5/roles/f9658866044e4fdeaddcc91781166faf => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:28:52.284221 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9da278-129c-454c-a3e1-d06b284817d3 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:52.284751 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9da278-129c-454c-a3e1-d06b284817d3 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:52.284865 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9da278-129c-454c-a3e1-d06b284817d3 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:52.286266 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 530/1078] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:52.292807 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2532a005-9782-47ba-96b5-49e9ec0c2f6f None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:52.293207 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2532a005-9782-47ba-96b5-49e9ec0c2f6f None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:52.293471 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2532a005-9782-47ba-96b5-49e9ec0c2f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:52.310798 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-2532a005-9782-47ba-96b5-49e9ec0c2f6f None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:52.311971 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-2532a005-9782-47ba-96b5-49e9ec0c2f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:52.319146 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:52.380802 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 549/1079] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:28:52.404386 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9da278-129c-454c-a3e1-d06b284817d3 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:52.423722 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:52.515591 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9da278-129c-454c-a3e1-d06b284817d3 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:52.554611 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb181ae-6f1f-495a-8837-c00f1f34b0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:52.555428 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb181ae-6f1f-495a-8837-c00f1f34b0bc None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:52.555860 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb181ae-6f1f-495a-8837-c00f1f34b0bc None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:52.556257 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb181ae-6f1f-495a-8837-c00f1f34b0bc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:52.567056 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb181ae-6f1f-495a-8837-c00f1f34b0bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:52.575745 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-2fb181ae-6f1f-495a-8837-c00f1f34b0bc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:28:52.575745 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:52.575745 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:52.575745 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:52.575745 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:52.575745 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:52.575745 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:52.575745 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:52.575745 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:52.575745 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:52.575745 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:52.575745 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:52.575745 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:52.575745 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:52.575745 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:52.575745 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:52.576684 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:52.576684 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:52.576684 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:52.576684 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:52.576684 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:52.576684 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:52.576684 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:52.576684 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:52.576684 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:52.576684 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:52.576684 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:52.576684 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:52.576684 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:52.576684 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:52.576684 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:52.576684 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:52.577527 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:52.577527 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:52.577527 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:52.577527 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:52.577527 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:52.577527 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:52.577527 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:52.577527 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:52.577527 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:52.577527 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:52.577527 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:28:52.577527 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:52.577527 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 531/1080] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Jun 24 08:28:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:52.583055 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2532a005-9782-47ba-96b5-49e9ec0c2f6f None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:52.684096 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2532a005-9782-47ba-96b5-49e9ec0c2f6f None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:52.719554 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa6307c-4b6a-48c6-8385-379d46729a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:52.720214 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6307c-4b6a-48c6-8385-379d46729a96 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:52.720491 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6307c-4b6a-48c6-8385-379d46729a96 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:52.720763 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6307c-4b6a-48c6-8385-379d46729a96 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:52.721497 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa6307c-4b6a-48c6-8385-379d46729a96 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:52.721902 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa6307c-4b6a-48c6-8385-379d46729a96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:52.723253 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa6307c-4b6a-48c6-8385-379d46729a96 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:52.731994 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 550/1081] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:28:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:52.737779 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb181ae-6f1f-495a-8837-c00f1f34b0bc None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:52.816079 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:52.839496 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb181ae-6f1f-495a-8837-c00f1f34b0bc None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:52.875627 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e85f89-4bdc-48ce-9777-838ed180fb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:52.876498 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e85f89-4bdc-48ce-9777-838ed180fb0d None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:52.876782 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e85f89-4bdc-48ce-9777-838ed180fb0d None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:52.877052 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e85f89-4bdc-48ce-9777-838ed180fb0d None admin] PATH_INFO: `/v3/users/4af9c25b61c949da9590c3ee0ea0b1cf` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:52.878505 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e85f89-4bdc-48ce-9777-838ed180fb0d None admin] RBAC: Authorizing `identity:get_user(user_id=4af9c25b61c949da9590c3ee0ea0b1cf)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:52.879935 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e85f89-4bdc-48ce-9777-838ed180fb0d None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:52.881565 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 532/1082] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Jun 24 08:28:52 2022] GET /identity/v3/users/4af9c25b61c949da9590c3ee0ea0b1cf => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:52.887866 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa6307c-4b6a-48c6-8385-379d46729a96 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:52.891689 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:52.993700 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa6307c-4b6a-48c6-8385-379d46729a96 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:53.031691 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6838cc-2ee8-49f6-97d8-635026217824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:53.032592 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6838cc-2ee8-49f6-97d8-635026217824 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:53.032900 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6838cc-2ee8-49f6-97d8-635026217824 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:53.033187 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6838cc-2ee8-49f6-97d8-635026217824 None admin] PATH_INFO: `/v3/projects/3997d3abea6b443b9df2e09c7ad2092e` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:53.034477 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6838cc-2ee8-49f6-97d8-635026217824 None admin] RBAC: Authorizing `identity:get_project(project_id=3997d3abea6b443b9df2e09c7ad2092e)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:53.035961 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6838cc-2ee8-49f6-97d8-635026217824 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:53.041810 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 551/1083] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:28:52 2022] GET /identity/v3/projects/3997d3abea6b443b9df2e09c7ad2092e => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:53.044889 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e85f89-4bdc-48ce-9777-838ed180fb0d None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:53.048981 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5733524e-980f-4edb-b779-d2acb66412c1 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:53.049273 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5733524e-980f-4edb-b779-d2acb66412c1 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:53.049548 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5733524e-980f-4edb-b779-d2acb66412c1 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:53.050935 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 552/1084] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:53.056188 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea86e567-423a-4162-b4c4-a08b14a8384c None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:53.056475 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea86e567-423a-4162-b4c4-a08b14a8384c None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:53.056756 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea86e567-423a-4162-b4c4-a08b14a8384c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:53.073613 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-ea86e567-423a-4162-b4c4-a08b14a8384c None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:53.074768 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-ea86e567-423a-4162-b4c4-a08b14a8384c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:53.146182 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 553/1085] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 08:28:53.159931 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e85f89-4bdc-48ce-9777-838ed180fb0d None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:53.170682 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea86e567-423a-4162-b4c4-a08b14a8384c None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:53.172829 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:53.200249 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc6bcb-356f-47ef-aed3-0e4cc1de86c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:53.201170 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc6bcb-356f-47ef-aed3-0e4cc1de86c6 None admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:53.201450 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc6bcb-356f-47ef-aed3-0e4cc1de86c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:53.201816 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc6bcb-356f-47ef-aed3-0e4cc1de86c6 None admin] PATH_INFO: `/v3/projects/3997d3abea6b443b9df2e09c7ad2092e/users/4af9c25b61c949da9590c3ee0ea0b1cf/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:53.203773 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bc6bcb-356f-47ef-aed3-0e4cc1de86c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=3997d3abea6b443b9df2e09c7ad2092e, user_id=4af9c25b61c949da9590c3ee0ea0b1cf, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:53.205210 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bc6bcb-356f-47ef-aed3-0e4cc1de86c6 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:53.220952 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 533/1086] 10.222.0.13 () {58 vars in 1439 bytes} [Fri Jun 24 08:28:53 2022] PUT /identity/v3/projects/3997d3abea6b443b9df2e09c7ad2092e/users/4af9c25b61c949da9590c3ee0ea0b1cf/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:28:53.294099 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea86e567-423a-4162-b4c4-a08b14a8384c None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:53.332188 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff2681-d280-4b1f-9867-5e388cf6661e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:53.332916 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff2681-d280-4b1f-9867-5e388cf6661e None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:53.333200 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff2681-d280-4b1f-9867-5e388cf6661e None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:53.333475 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff2681-d280-4b1f-9867-5e388cf6661e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:53.346505 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ff2681-d280-4b1f-9867-5e388cf6661e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:53.355657 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-44ff2681-d280-4b1f-9867-5e388cf6661e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 08:28:53.355657 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:53.355657 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:53.355657 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:53.355657 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:53.355657 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:53.355657 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:53.355657 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:53.355657 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:53.355657 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:53.355657 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:53.355657 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:53.355657 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:53.355657 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:53.355657 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:53.355657 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:53.356747 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:53.356747 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:53.356747 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:53.356747 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:53.356747 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:53.356747 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:53.356747 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:53.356747 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:53.356747 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:53.356747 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:53.356747 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:53.356747 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:53.356747 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:53.356747 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:53.356747 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:53.356747 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:53.357592 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:53.357592 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:53.357592 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:53.357592 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:53.357592 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:53.357592 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:53.357592 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:53.357592 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:53.357592 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:53.357592 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:53.357592 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 08:28:53.357592 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:53.357592 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 554/1087] 10.222.0.13 () {56 vars in 1120 bytes} [Fri Jun 24 08:28:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:28:53.363169 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc6bcb-356f-47ef-aed3-0e4cc1de86c6 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:53.369889 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:53.469725 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc6bcb-356f-47ef-aed3-0e4cc1de86c6 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:53.503108 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b32ee05-d543-4995-b691-1ffa600a8934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:53.504025 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b32ee05-d543-4995-b691-1ffa600a8934 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:53.504308 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b32ee05-d543-4995-b691-1ffa600a8934 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:53.504583 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b32ee05-d543-4995-b691-1ffa600a8934 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:53.505321 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b32ee05-d543-4995-b691-1ffa600a8934 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:53.505624 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b32ee05-d543-4995-b691-1ffa600a8934 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:53.507017 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b32ee05-d543-4995-b691-1ffa600a8934 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:53.516775 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 534/1088] 10.222.0.13 () {56 vars in 1115 bytes} [Fri Jun 24 08:28:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:53.522537 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff2681-d280-4b1f-9867-5e388cf6661e None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:53.583254 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:53.622268 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff2681-d280-4b1f-9867-5e388cf6661e None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:53.658972 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299c6d69-5c11-4f16-ba1e-726e68a70e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:53.659760 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c6d69-5c11-4f16-ba1e-726e68a70e86 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:53.660041 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c6d69-5c11-4f16-ba1e-726e68a70e86 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:53.660318 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c6d69-5c11-4f16-ba1e-726e68a70e86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:53.673926 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299c6d69-5c11-4f16-ba1e-726e68a70e86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:53.675368 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299c6d69-5c11-4f16-ba1e-726e68a70e86 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:53.692819 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-299c6d69-5c11-4f16-ba1e-726e68a70e86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:53.692819 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:53.692819 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:53.692819 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:53.692819 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:53.692819 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:53.692819 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:53.692819 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:53.692819 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:53.692819 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:53.692819 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:53.692819 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:53.692819 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:28:53.692819 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:28:53.692819 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:28:53.693857 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:28:53.693857 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:53.693857 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:53.693857 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:53.693857 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:53.693857 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:53.693857 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:53.693857 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:53.693857 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:53.693857 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:53.693857 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:53.693857 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:53.693857 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:53.693857 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:53.693857 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:53.693857 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:53.693857 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:53.694758 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:28:53.694758 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:28:53.694758 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:53.694758 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:53.694758 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 555/1089] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:28:53.700339 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b32ee05-d543-4995-b691-1ffa600a8934 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:53.765379 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:53.799419 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b32ee05-d543-4995-b691-1ffa600a8934 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:53.832932 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edae867c-8a71-4345-bab7-e23ff8611168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:53.833638 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae867c-8a71-4345-bab7-e23ff8611168 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:53.833903 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae867c-8a71-4345-bab7-e23ff8611168 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:53.834179 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae867c-8a71-4345-bab7-e23ff8611168 None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:53.834891 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edae867c-8a71-4345-bab7-e23ff8611168 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:53.835196 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edae867c-8a71-4345-bab7-e23ff8611168 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:53.836668 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edae867c-8a71-4345-bab7-e23ff8611168 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:53.848531 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 535/1090] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:28:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:53.854579 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299c6d69-5c11-4f16-ba1e-726e68a70e86 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:53.964475 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299c6d69-5c11-4f16-ba1e-726e68a70e86 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:53.978585 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:53.999979 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa11fa-6633-4d0a-ba11-ced26bbb8928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:54.000763 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa11fa-6633-4d0a-ba11-ced26bbb8928 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:54.001044 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa11fa-6633-4d0a-ba11-ced26bbb8928 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:54.001315 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa11fa-6633-4d0a-ba11-ced26bbb8928 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:54.009940 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fa11fa-6633-4d0a-ba11-ced26bbb8928 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:54.011375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fa11fa-6633-4d0a-ba11-ced26bbb8928 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:54.024917 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-d0fa11fa-6633-4d0a-ba11-ced26bbb8928 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 08:28:54.024917 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:54.024917 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:54.024917 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:54.024917 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:54.024917 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:54.024917 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:54.024917 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:54.024917 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:54.024917 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:54.024917 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:54.024917 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:54.024917 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:28:54.024917 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:28:54.024917 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:28:54.024917 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:28:54.025900 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:54.025900 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:54.025900 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:28:54.025900 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:28:54.025900 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:28:54.025900 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:28:54.025900 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:54.025900 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:54.025900 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:54.025900 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:54.025900 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:54.025900 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:54.025900 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:54.025900 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:54.025900 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:54.025900 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:54.025900 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:54.025900 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:54.027193 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:54.027193 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:54.027193 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:28:54.027193 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:28:54.027193 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:28:54.027193 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:28:54.027193 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:28:54.027193 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:28:54.027193 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 08:28:54.027193 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:28:54.027193 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 556/1091] 10.222.0.13 () {56 vars in 1093 bytes} [Fri Jun 24 08:28:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:54.031890 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edae867c-8a71-4345-bab7-e23ff8611168 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:54.134645 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edae867c-8a71-4345-bab7-e23ff8611168 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:54.159134 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:54.168830 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f7c602-f422-4310-b63f-7f371f16118c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:54.169492 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7c602-f422-4310-b63f-7f371f16118c None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:54.169804 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7c602-f422-4310-b63f-7f371f16118c None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:54.170083 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7c602-f422-4310-b63f-7f371f16118c None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:54.170919 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f7c602-f422-4310-b63f-7f371f16118c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:54.171231 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f7c602-f422-4310-b63f-7f371f16118c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:54.172664 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f7c602-f422-4310-b63f-7f371f16118c None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:54.190445 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 536/1092] 10.222.0.13 () {56 vars in 1133 bytes} [Fri Jun 24 08:28:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:54.196345 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa11fa-6633-4d0a-ba11-ced26bbb8928 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:54.308433 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa11fa-6633-4d0a-ba11-ced26bbb8928 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:54.342114 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd8848c-51c3-4d26-9995-490399ad7013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:54.342775 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd8848c-51c3-4d26-9995-490399ad7013 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:54.343062 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd8848c-51c3-4d26-9995-490399ad7013 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:54.343342 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd8848c-51c3-4d26-9995-490399ad7013 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:54.391529 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:54.417194 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd8848c-51c3-4d26-9995-490399ad7013 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:54.418658 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd8848c-51c3-4d26-9995-490399ad7013 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:54.521602 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 557/1093] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:28:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 24 08:28:54.528092 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f7c602-f422-4310-b63f-7f371f16118c None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:54.630576 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f7c602-f422-4310-b63f-7f371f16118c None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:54.665362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18913e39-e196-434b-a6d4-a1c2f326bb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:54.666103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18913e39-e196-434b-a6d4-a1c2f326bb9a None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:54.666432 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18913e39-e196-434b-a6d4-a1c2f326bb9a None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:54.666647 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18913e39-e196-434b-a6d4-a1c2f326bb9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:54.676490 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18913e39-e196-434b-a6d4-a1c2f326bb9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:54.677957 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18913e39-e196-434b-a6d4-a1c2f326bb9a None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:54.695588 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-18913e39-e196-434b-a6d4-a1c2f326bb9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:54.695588 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:54.695588 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:54.695588 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:54.695588 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:54.695588 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:54.695588 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:54.695588 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:54.695588 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:54.695588 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:54.695588 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:54.695588 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:54.695588 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:28:54.695588 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:28:54.695588 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:28:54.696529 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:28:54.696529 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:54.696529 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:54.696529 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:54.696529 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:54.696529 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:54.696529 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:54.696529 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:54.696529 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:54.696529 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:54.696529 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:54.696529 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:54.696529 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:54.696529 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:54.696529 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:54.696529 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:54.696529 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:54.697558 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:28:54.697558 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:28:54.697558 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:54.697558 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:54.697558 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 537/1094] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:54.702657 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd8848c-51c3-4d26-9995-490399ad7013 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:54.775794 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:54.800953 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd8848c-51c3-4d26-9995-490399ad7013 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:54.835627 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb97e0f-5905-4f39-a471-e8072ead13d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:54.836287 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb97e0f-5905-4f39-a471-e8072ead13d7 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:54.836579 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb97e0f-5905-4f39-a471-e8072ead13d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:54.837412 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb97e0f-5905-4f39-a471-e8072ead13d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:54.838171 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb97e0f-5905-4f39-a471-e8072ead13d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:54.838506 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb97e0f-5905-4f39-a471-e8072ead13d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:54.839842 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb97e0f-5905-4f39-a471-e8072ead13d7 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:54.851120 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 558/1095] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:28:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:54.856779 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18913e39-e196-434b-a6d4-a1c2f326bb9a None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:54.902070 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:54.965111 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18913e39-e196-434b-a6d4-a1c2f326bb9a None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:54.997900 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a20593-4e2e-4418-90e7-fa4655789d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:54.998684 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a20593-4e2e-4418-90e7-fa4655789d2c None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:54.998967 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a20593-4e2e-4418-90e7-fa4655789d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:54.999242 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a20593-4e2e-4418-90e7-fa4655789d2c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:55.006285 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a20593-4e2e-4418-90e7-fa4655789d2c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:55.007725 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a20593-4e2e-4418-90e7-fa4655789d2c None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:55.021507 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-04a20593-4e2e-4418-90e7-fa4655789d2c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:28:55.021507 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:55.021507 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:55.021507 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:55.021507 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:55.021507 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:55.021507 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:55.021507 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:55.021507 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:55.021507 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:55.021507 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:55.021507 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:55.021507 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:28:55.021507 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:28:55.021507 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:28:55.022582 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:28:55.022582 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:55.022582 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:55.022582 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:55.022582 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:55.022582 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:55.022582 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:55.022582 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:55.022582 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:55.022582 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:55.022582 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:55.022582 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:55.022582 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:55.022582 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:55.022582 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:55.022582 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:55.022582 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:55.023465 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:28:55.023465 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:28:55.023465 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:28:55.023465 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:28:55.023465 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:28:55.023465 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:28:55.023465 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:28:55.023465 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:55.023465 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 538/1096] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:28:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:55.029511 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb97e0f-5905-4f39-a471-e8072ead13d7 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:55.136625 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb97e0f-5905-4f39-a471-e8072ead13d7 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:55.169708 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cea298-0efc-4aa9-bc66-512805394b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:55.170367 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cea298-0efc-4aa9-bc66-512805394b14 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:55.170653 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cea298-0efc-4aa9-bc66-512805394b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:55.170932 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cea298-0efc-4aa9-bc66-512805394b14 None admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:55.171716 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cea298-0efc-4aa9-bc66-512805394b14 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:55.172027 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cea298-0efc-4aa9-bc66-512805394b14 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:55.173454 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cea298-0efc-4aa9-bc66-512805394b14 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:55.175110 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:55.187432 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 559/1097] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Jun 24 08:28:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:55.193626 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a20593-4e2e-4418-90e7-fa4655789d2c None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:55.288501 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:55.294842 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a20593-4e2e-4418-90e7-fa4655789d2c None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:55.319681 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80d0fb-58bd-40fb-aba1-66f9c7eb64f5 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:55.320192 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80d0fb-58bd-40fb-aba1-66f9c7eb64f5 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:55.320284 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80d0fb-58bd-40fb-aba1-66f9c7eb64f5 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:55.322362 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 560/1098] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:28:55.327171 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d241a-f014-4cd9-b28e-12808605ba7f None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:55.327454 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d241a-f014-4cd9-b28e-12808605ba7f None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:55.327727 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d241a-f014-4cd9-b28e-12808605ba7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:55.329590 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed7747e-7754-466c-99e2-923a9ec7b332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:55.330267 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed7747e-7754-466c-99e2-923a9ec7b332 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:55.330550 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed7747e-7754-466c-99e2-923a9ec7b332 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:55.330822 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed7747e-7754-466c-99e2-923a9ec7b332 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:55.331645 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed7747e-7754-466c-99e2-923a9ec7b332 None admin] RBAC: Adding query filter params (role.id=f9658866044e4fdeaddcc91781166faf, scope.project.id=180b0af5863349e99ae412b07c1e9465, user.id=74bb1bc6821b4500b26f160ccae5b7d5) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:55.331950 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed7747e-7754-466c-99e2-923a9ec7b332 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:55.333366 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed7747e-7754-466c-99e2-923a9ec7b332 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:55.339355 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 539/1099] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Jun 24 08:28:55 2022] GET /identity/v3/role_assignments?user.id=74bb1bc6821b4500b26f160ccae5b7d5&scope.project.id=180b0af5863349e99ae412b07c1e9465&role.id=f9658866044e4fdeaddcc91781166faf => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:55.344399 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-6c7d241a-f014-4cd9-b28e-12808605ba7f None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:55.345607 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-6c7d241a-f014-4cd9-b28e-12808605ba7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:55.421065 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 561/1100] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 08:28:55.445343 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed7747e-7754-466c-99e2-923a9ec7b332 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:55.550115 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed7747e-7754-466c-99e2-923a9ec7b332 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:55.584087 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01db9b-24a7-4ec8-9143-17c2c61f0c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:55.584876 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01db9b-24a7-4ec8-9143-17c2c61f0c8c None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:55.585162 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01db9b-24a7-4ec8-9143-17c2c61f0c8c None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:55.585440 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01db9b-24a7-4ec8-9143-17c2c61f0c8c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:55.596211 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01db9b-24a7-4ec8-9143-17c2c61f0c8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:55.604788 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-3c01db9b-24a7-4ec8-9143-17c2c61f0c8c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:28:55.604788 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:55.604788 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:55.604788 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:55.604788 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:55.604788 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:55.604788 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:55.604788 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:55.604788 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:55.604788 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:55.604788 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:55.604788 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:55.604788 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:55.604788 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:55.604788 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:55.604788 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:55.605811 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:55.605811 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:55.605811 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:55.605811 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:55.605811 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:55.605811 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:55.605811 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:55.605811 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:55.605811 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:55.605811 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:55.605811 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:55.605811 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:55.605811 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:55.605811 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:55.605811 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:55.605811 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:55.606638 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:55.606638 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:55.606638 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:55.606638 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:55.606638 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:55.606638 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:55.606638 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:55.606638 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:55.606638 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:55.606638 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:55.606638 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:28:55.606638 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:28:55.606638 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 540/1101] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Jun 24 08:28:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:55.612105 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d241a-f014-4cd9-b28e-12808605ba7f None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:55.716372 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d241a-f014-4cd9-b28e-12808605ba7f None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:55.741589 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:55.754012 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da10b2-d7a5-4c4c-b4d4-e3415143f9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:55.754691 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da10b2-d7a5-4c4c-b4d4-e3415143f9c9 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:55.754974 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da10b2-d7a5-4c4c-b4d4-e3415143f9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:55.755247 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da10b2-d7a5-4c4c-b4d4-e3415143f9c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:55.756003 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50da10b2-d7a5-4c4c-b4d4-e3415143f9c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:55.756307 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50da10b2-d7a5-4c4c-b4d4-e3415143f9c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:55.757670 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50da10b2-d7a5-4c4c-b4d4-e3415143f9c9 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:55.766626 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 562/1102] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:28:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:55.772539 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01db9b-24a7-4ec8-9143-17c2c61f0c8c None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:55.796381 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:55.878853 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01db9b-24a7-4ec8-9143-17c2c61f0c8c None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:55.918209 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c100a0-9fb8-4aba-bd03-5b0991806a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:55.919090 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c100a0-9fb8-4aba-bd03-5b0991806a09 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:55.919370 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c100a0-9fb8-4aba-bd03-5b0991806a09 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:55.919653 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c100a0-9fb8-4aba-bd03-5b0991806a09 None admin] PATH_INFO: `/v3/users/4af9c25b61c949da9590c3ee0ea0b1cf` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:55.920867 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c100a0-9fb8-4aba-bd03-5b0991806a09 None admin] RBAC: Authorizing `identity:get_user(user_id=4af9c25b61c949da9590c3ee0ea0b1cf)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:55.922417 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c100a0-9fb8-4aba-bd03-5b0991806a09 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:55.924086 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 541/1103] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Jun 24 08:28:55 2022] GET /identity/v3/users/4af9c25b61c949da9590c3ee0ea0b1cf => generated 285 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:55.930168 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da10b2-d7a5-4c4c-b4d4-e3415143f9c9 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:56.034588 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da10b2-d7a5-4c4c-b4d4-e3415143f9c9 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:56.071666 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6195677-bb69-461f-9e04-d7d89712af64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:56.072322 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6195677-bb69-461f-9e04-d7d89712af64 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:56.072612 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6195677-bb69-461f-9e04-d7d89712af64 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:56.072888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6195677-bb69-461f-9e04-d7d89712af64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:56.128540 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:56.143386 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6195677-bb69-461f-9e04-d7d89712af64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:56.144763 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6195677-bb69-461f-9e04-d7d89712af64 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:56.251243 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 563/1104] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:28:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 08:28:56.257479 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c100a0-9fb8-4aba-bd03-5b0991806a09 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:56.346095 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:56.358251 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c100a0-9fb8-4aba-bd03-5b0991806a09 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:56.391802 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9596c1a1-8eb9-47ce-83b0-cde647862362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:56.392645 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9596c1a1-8eb9-47ce-83b0-cde647862362 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:56.392930 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9596c1a1-8eb9-47ce-83b0-cde647862362 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:56.393206 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9596c1a1-8eb9-47ce-83b0-cde647862362 None admin] PATH_INFO: `/v3/projects/3997d3abea6b443b9df2e09c7ad2092e` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:56.394444 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9596c1a1-8eb9-47ce-83b0-cde647862362 None admin] RBAC: Authorizing `identity:get_project(project_id=3997d3abea6b443b9df2e09c7ad2092e)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:56.395832 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9596c1a1-8eb9-47ce-83b0-cde647862362 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:56.397535 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 542/1105] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:28:56 2022] GET /identity/v3/projects/3997d3abea6b443b9df2e09c7ad2092e => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:56.404114 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6195677-bb69-461f-9e04-d7d89712af64 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:56.484692 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:56.499152 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6195677-bb69-461f-9e04-d7d89712af64 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:56.532717 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073b3854-1c3c-421f-ac72-a1a92e9a7752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:56.533364 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073b3854-1c3c-421f-ac72-a1a92e9a7752 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:56.533650 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073b3854-1c3c-421f-ac72-a1a92e9a7752 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:56.533963 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073b3854-1c3c-421f-ac72-a1a92e9a7752 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:56.534788 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073b3854-1c3c-421f-ac72-a1a92e9a7752 None admin] RBAC: Adding query filter params (role.id=acfe707eee6944ad94d1915fc9ee7ea3, scope.project.id=3997d3abea6b443b9df2e09c7ad2092e, user.id=4af9c25b61c949da9590c3ee0ea0b1cf) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:56.535100 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073b3854-1c3c-421f-ac72-a1a92e9a7752 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:56.536518 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073b3854-1c3c-421f-ac72-a1a92e9a7752 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:56.542377 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 564/1106] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Jun 24 08:28:56 2022] GET /identity/v3/role_assignments?user.id=4af9c25b61c949da9590c3ee0ea0b1cf&scope.project.id=3997d3abea6b443b9df2e09c7ad2092e&role.id=acfe707eee6944ad94d1915fc9ee7ea3 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:58.620354 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7091aeb8-5767-4854-911f-f45e43a251db None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:58.620949 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7091aeb8-5767-4854-911f-f45e43a251db None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:58.620949 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7091aeb8-5767-4854-911f-f45e43a251db None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:58.622375 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 543/1107] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:28:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:58.627563 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9551007-fe24-4865-bdfb-6051fc1ad8c6 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:58.627996 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9551007-fe24-4865-bdfb-6051fc1ad8c6 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:58.628119 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9551007-fe24-4865-bdfb-6051fc1ad8c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:58.645393 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-f9551007-fe24-4865-bdfb-6051fc1ad8c6 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:58.646606 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-f9551007-fe24-4865-bdfb-6051fc1ad8c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:58.713126 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 565/1108] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:28:58.740914 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7091aeb8-5767-4854-911f-f45e43a251db None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:58.842092 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7091aeb8-5767-4854-911f-f45e43a251db None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:58.877053 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82092b07-f9f7-4ef9-adf6-1ee3bd4f4668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:58.877793 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82092b07-f9f7-4ef9-adf6-1ee3bd4f4668 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:58.878081 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82092b07-f9f7-4ef9-adf6-1ee3bd4f4668 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:58.878359 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82092b07-f9f7-4ef9-adf6-1ee3bd4f4668 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:58.879785 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82092b07-f9f7-4ef9-adf6-1ee3bd4f4668 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:58.881215 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82092b07-f9f7-4ef9-adf6-1ee3bd4f4668 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:58.882864 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 544/1109] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:58.888556 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9551007-fe24-4865-bdfb-6051fc1ad8c6 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:58.974579 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:58.993804 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9551007-fe24-4865-bdfb-6051fc1ad8c6 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:59.026889 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b3de58-5b16-425e-afe7-523ba9725f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:59.027546 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3de58-5b16-425e-afe7-523ba9725f76 None admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:59.027832 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3de58-5b16-425e-afe7-523ba9725f76 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:59.028109 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3de58-5b16-425e-afe7-523ba9725f76 None admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:59.028817 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b3de58-5b16-425e-afe7-523ba9725f76 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:59.030274 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b3de58-5b16-425e-afe7-523ba9725f76 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:59.049952 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 566/1110] 10.222.0.13 () {60 vars in 1132 bytes} [Fri Jun 24 08:28:58 2022] POST /identity/v3/users => generated 286 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:29:01.093567 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf6329a-f0fa-44d2-8eda-9b19a86f44a4 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:01.094149 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf6329a-f0fa-44d2-8eda-9b19a86f44a4 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:01.094149 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf6329a-f0fa-44d2-8eda-9b19a86f44a4 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:01.095504 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 545/1111] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:29:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:01.100813 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dab827-7dff-4164-a942-accff4c7a31b None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:01.101199 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dab827-7dff-4164-a942-accff4c7a31b None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:01.101363 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dab827-7dff-4164-a942-accff4c7a31b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:01.118319 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-31dab827-7dff-4164-a942-accff4c7a31b None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:01.119581 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-31dab827-7dff-4164-a942-accff4c7a31b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:01.184766 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 567/1112] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:29:01.207785 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf6329a-f0fa-44d2-8eda-9b19a86f44a4 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:01.233223 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:01.307175 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf6329a-f0fa-44d2-8eda-9b19a86f44a4 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:01.339619 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160216ad-63d3-4a04-93fd-4f9799481765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:01.340461 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160216ad-63d3-4a04-93fd-4f9799481765 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:01.340748 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160216ad-63d3-4a04-93fd-4f9799481765 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:01.341025 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160216ad-63d3-4a04-93fd-4f9799481765 None admin] PATH_INFO: `/v3/roles/d6ada62a74a94222b864a6c283f8d9b1` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:01.342531 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160216ad-63d3-4a04-93fd-4f9799481765 None admin] RBAC: Authorizing `identity:get_role(role_id=d6ada62a74a94222b864a6c283f8d9b1)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:01.343960 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160216ad-63d3-4a04-93fd-4f9799481765 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:01.345285 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 546/1113] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Jun 24 08:29:01 2022] GET /identity/v3/roles/d6ada62a74a94222b864a6c283f8d9b1 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:01.351058 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dab827-7dff-4164-a942-accff4c7a31b None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:01.415628 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:01.449976 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dab827-7dff-4164-a942-accff4c7a31b None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:01.483262 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68613914-e66b-4929-90c6-900589a66c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:01.484112 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68613914-e66b-4929-90c6-900589a66c6c None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:01.484390 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68613914-e66b-4929-90c6-900589a66c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:01.484661 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68613914-e66b-4929-90c6-900589a66c6c None admin] PATH_INFO: `/v3/users/b51e0e5be73a4c7aac122a2e66d5a326` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:01.497208 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68613914-e66b-4929-90c6-900589a66c6c None admin] RBAC: Authorizing `identity:get_user(user_id=b51e0e5be73a4c7aac122a2e66d5a326)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:01.498650 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68613914-e66b-4929-90c6-900589a66c6c None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:01.500270 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 568/1114] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Jun 24 08:29:01 2022] GET /identity/v3/users/b51e0e5be73a4c7aac122a2e66d5a326 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:01.506197 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160216ad-63d3-4a04-93fd-4f9799481765 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:01.594946 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:01.604296 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160216ad-63d3-4a04-93fd-4f9799481765 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:01.638289 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c2fa2-e977-4ac5-8e6b-aa0e77a7bab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:01.638927 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c2fa2-e977-4ac5-8e6b-aa0e77a7bab4 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:01.639213 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c2fa2-e977-4ac5-8e6b-aa0e77a7bab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:01.639490 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c2fa2-e977-4ac5-8e6b-aa0e77a7bab4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:01.705135 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29c2fa2-e977-4ac5-8e6b-aa0e77a7bab4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:01.706602 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29c2fa2-e977-4ac5-8e6b-aa0e77a7bab4 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:01.794543 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:01.804229 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 547/1115] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:29:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 08:29:01.810884 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68613914-e66b-4929-90c6-900589a66c6c None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:01.908084 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68613914-e66b-4929-90c6-900589a66c6c None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:01.932060 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:01.942638 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c13265-8239-4642-8358-f3215e3a0d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:01.943490 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c13265-8239-4642-8358-f3215e3a0d0b None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:01.943772 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c13265-8239-4642-8358-f3215e3a0d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:01.944047 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c13265-8239-4642-8358-f3215e3a0d0b None admin] PATH_INFO: `/v3/projects/3997d3abea6b443b9df2e09c7ad2092e` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:01.945217 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c13265-8239-4642-8358-f3215e3a0d0b None admin] RBAC: Authorizing `identity:get_project(project_id=3997d3abea6b443b9df2e09c7ad2092e)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:01.946677 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c13265-8239-4642-8358-f3215e3a0d0b None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:01.948368 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 569/1116] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:29:01 2022] GET /identity/v3/projects/3997d3abea6b443b9df2e09c7ad2092e => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:01.954431 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c2fa2-e977-4ac5-8e6b-aa0e77a7bab4 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:02.069380 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c2fa2-e977-4ac5-8e6b-aa0e77a7bab4 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:02.101792 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705d0970-13f0-45f4-b691-eb4a7886c201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:02.103023 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d0970-13f0-45f4-b691-eb4a7886c201 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:02.103299 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d0970-13f0-45f4-b691-eb4a7886c201 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:02.103608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d0970-13f0-45f4-b691-eb4a7886c201 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:02.104486 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705d0970-13f0-45f4-b691-eb4a7886c201 None admin] RBAC: Adding query filter params (role.id=d6ada62a74a94222b864a6c283f8d9b1, scope.project.id=3997d3abea6b443b9df2e09c7ad2092e, user.id=b51e0e5be73a4c7aac122a2e66d5a326) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:02.104826 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705d0970-13f0-45f4-b691-eb4a7886c201 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:02.106541 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705d0970-13f0-45f4-b691-eb4a7886c201 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:02.112111 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 548/1117] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Jun 24 08:29:01 2022] GET /identity/v3/role_assignments?user.id=b51e0e5be73a4c7aac122a2e66d5a326&scope.project.id=3997d3abea6b443b9df2e09c7ad2092e&role.id=d6ada62a74a94222b864a6c283f8d9b1 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:04.115706 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c266f3b-2292-4770-8917-1496b98a9644 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:04.116287 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c266f3b-2292-4770-8917-1496b98a9644 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:04.116287 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c266f3b-2292-4770-8917-1496b98a9644 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:04.117609 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 570/1118] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:29:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:04.123458 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a05a7-93eb-43c8-a2ae-fff08c656470 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:04.123846 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a05a7-93eb-43c8-a2ae-fff08c656470 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:04.124019 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a05a7-93eb-43c8-a2ae-fff08c656470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:04.141933 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-2e9a05a7-93eb-43c8-a2ae-fff08c656470 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:04.143131 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-2e9a05a7-93eb-43c8-a2ae-fff08c656470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:04.194494 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:04.212421 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 549/1119] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:29:04.235022 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c266f3b-2292-4770-8917-1496b98a9644 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:04.337635 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c266f3b-2292-4770-8917-1496b98a9644 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:04.375414 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c962cb2-f6f0-4ab0-9a8f-0b0b406a8100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:04.376271 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c962cb2-f6f0-4ab0-9a8f-0b0b406a8100 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:04.376560 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c962cb2-f6f0-4ab0-9a8f-0b0b406a8100 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:04.376841 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c962cb2-f6f0-4ab0-9a8f-0b0b406a8100 None admin] PATH_INFO: `/v3/roles/d6ada62a74a94222b864a6c283f8d9b1` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:04.378360 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c962cb2-f6f0-4ab0-9a8f-0b0b406a8100 None admin] RBAC: Authorizing `identity:get_role(role_id=d6ada62a74a94222b864a6c283f8d9b1)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:04.379988 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c962cb2-f6f0-4ab0-9a8f-0b0b406a8100 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:04.381371 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 571/1120] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Jun 24 08:29:04 2022] GET /identity/v3/roles/d6ada62a74a94222b864a6c283f8d9b1 => generated 223 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:04.387844 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9a05a7-93eb-43c8-a2ae-fff08c656470 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:04.491843 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9a05a7-93eb-43c8-a2ae-fff08c656470 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:04.525301 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59db356-7ab8-4c53-a139-35cd92c1d436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:04.526175 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59db356-7ab8-4c53-a139-35cd92c1d436 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:04.526457 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59db356-7ab8-4c53-a139-35cd92c1d436 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:04.526728 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59db356-7ab8-4c53-a139-35cd92c1d436 None admin] PATH_INFO: `/v3/users/b51e0e5be73a4c7aac122a2e66d5a326` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:04.527883 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59db356-7ab8-4c53-a139-35cd92c1d436 None admin] RBAC: Authorizing `identity:get_user(user_id=b51e0e5be73a4c7aac122a2e66d5a326)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:04.529306 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59db356-7ab8-4c53-a139-35cd92c1d436 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:04.530961 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 550/1121] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Jun 24 08:29:04 2022] GET /identity/v3/users/b51e0e5be73a4c7aac122a2e66d5a326 => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:04.536807 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c962cb2-f6f0-4ab0-9a8f-0b0b406a8100 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:04.595002 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:04.635582 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c962cb2-f6f0-4ab0-9a8f-0b0b406a8100 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:04.668974 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af314587-b62e-4a2d-97d4-f2b5589f40fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:04.669836 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af314587-b62e-4a2d-97d4-f2b5589f40fe None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:04.670120 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af314587-b62e-4a2d-97d4-f2b5589f40fe None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:04.670391 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af314587-b62e-4a2d-97d4-f2b5589f40fe None admin] PATH_INFO: `/v3/projects/3997d3abea6b443b9df2e09c7ad2092e` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:04.671550 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af314587-b62e-4a2d-97d4-f2b5589f40fe None admin] RBAC: Authorizing `identity:get_project(project_id=3997d3abea6b443b9df2e09c7ad2092e)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:04.672951 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af314587-b62e-4a2d-97d4-f2b5589f40fe None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:04.674643 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 572/1122] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:29:04 2022] GET /identity/v3/projects/3997d3abea6b443b9df2e09c7ad2092e => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:04.680731 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59db356-7ab8-4c53-a139-35cd92c1d436 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:04.734572 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:04.779035 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59db356-7ab8-4c53-a139-35cd92c1d436 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:04.811694 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3d3745-0fbd-4cd7-9375-08e13db95593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:04.812565 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d3745-0fbd-4cd7-9375-08e13db95593 None admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:04.812849 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d3745-0fbd-4cd7-9375-08e13db95593 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:04.813122 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d3745-0fbd-4cd7-9375-08e13db95593 None admin] PATH_INFO: `/v3/projects/3997d3abea6b443b9df2e09c7ad2092e/users/b51e0e5be73a4c7aac122a2e66d5a326/roles/d6ada62a74a94222b864a6c283f8d9b1` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:04.815345 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3d3745-0fbd-4cd7-9375-08e13db95593 None admin] RBAC: Authorizing `identity:create_grant(project_id=3997d3abea6b443b9df2e09c7ad2092e, user_id=b51e0e5be73a4c7aac122a2e66d5a326, role_id=d6ada62a74a94222b864a6c283f8d9b1)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:04.816814 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3d3745-0fbd-4cd7-9375-08e13db95593 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:04.830579 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 551/1123] 10.222.0.13 () {58 vars in 1439 bytes} [Fri Jun 24 08:29:04 2022] PUT /identity/v3/projects/3997d3abea6b443b9df2e09c7ad2092e/users/b51e0e5be73a4c7aac122a2e66d5a326/roles/d6ada62a74a94222b864a6c283f8d9b1 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:29:06.823224 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105bdb3f-6f6a-49ac-8ed3-ebfe1f924295 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:06.823779 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105bdb3f-6f6a-49ac-8ed3-ebfe1f924295 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:06.823779 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105bdb3f-6f6a-49ac-8ed3-ebfe1f924295 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:06.825131 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 573/1124] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:29:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:06.830334 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9c7e25-ddd0-4a63-85e0-fffb83615431 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:06.830718 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9c7e25-ddd0-4a63-85e0-fffb83615431 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:06.830888 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9c7e25-ddd0-4a63-85e0-fffb83615431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:06.847825 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-6c9c7e25-ddd0-4a63-85e0-fffb83615431 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:06.848987 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-6c9c7e25-ddd0-4a63-85e0-fffb83615431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:06.912585 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 552/1125] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:29:06.935682 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105bdb3f-6f6a-49ac-8ed3-ebfe1f924295 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:07.034238 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105bdb3f-6f6a-49ac-8ed3-ebfe1f924295 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:07.060047 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:07.067890 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c4f794-de84-41a2-9bc0-a2d40fa2018c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:07.068727 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c4f794-de84-41a2-9bc0-a2d40fa2018c None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:07.069011 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c4f794-de84-41a2-9bc0-a2d40fa2018c None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:07.069285 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c4f794-de84-41a2-9bc0-a2d40fa2018c None admin] PATH_INFO: `/v3/roles/d6ada62a74a94222b864a6c283f8d9b1` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:07.070766 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c4f794-de84-41a2-9bc0-a2d40fa2018c None admin] RBAC: Authorizing `identity:get_role(role_id=d6ada62a74a94222b864a6c283f8d9b1)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:07.073184 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c4f794-de84-41a2-9bc0-a2d40fa2018c None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:07.074616 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 574/1126] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Jun 24 08:29:06 2022] GET /identity/v3/roles/d6ada62a74a94222b864a6c283f8d9b1 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:07.080242 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9c7e25-ddd0-4a63-85e0-fffb83615431 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:07.126324 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:07.179904 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9c7e25-ddd0-4a63-85e0-fffb83615431 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:07.215558 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5178da44-04df-4a52-81c5-db8bd329d65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:07.216404 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5178da44-04df-4a52-81c5-db8bd329d65e None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:07.216684 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5178da44-04df-4a52-81c5-db8bd329d65e None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:07.216957 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5178da44-04df-4a52-81c5-db8bd329d65e None admin] PATH_INFO: `/v3/users/b51e0e5be73a4c7aac122a2e66d5a326` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:07.218202 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5178da44-04df-4a52-81c5-db8bd329d65e None admin] RBAC: Authorizing `identity:get_user(user_id=b51e0e5be73a4c7aac122a2e66d5a326)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:07.219623 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5178da44-04df-4a52-81c5-db8bd329d65e None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:07.221235 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 553/1127] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Jun 24 08:29:07 2022] GET /identity/v3/users/b51e0e5be73a4c7aac122a2e66d5a326 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:07.226957 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c4f794-de84-41a2-9bc0-a2d40fa2018c None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:07.325083 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c4f794-de84-41a2-9bc0-a2d40fa2018c None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:07.362320 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c6fa29-d5a5-4979-91e3-97e11ea72832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:07.362961 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c6fa29-d5a5-4979-91e3-97e11ea72832 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:07.363244 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c6fa29-d5a5-4979-91e3-97e11ea72832 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:07.363519 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c6fa29-d5a5-4979-91e3-97e11ea72832 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:07.423410 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:07.430419 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c6fa29-d5a5-4979-91e3-97e11ea72832 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:07.431879 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c6fa29-d5a5-4979-91e3-97e11ea72832 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:07.527596 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 575/1128] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:29:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 08:29:07.533602 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5178da44-04df-4a52-81c5-db8bd329d65e None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:07.628708 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5178da44-04df-4a52-81c5-db8bd329d65e None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:07.652174 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:07.673729 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb4912f-a88b-4816-bc7d-d91a8bf1c1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:07.674615 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4912f-a88b-4816-bc7d-d91a8bf1c1da None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:07.674899 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4912f-a88b-4816-bc7d-d91a8bf1c1da None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:07.675374 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4912f-a88b-4816-bc7d-d91a8bf1c1da None admin] PATH_INFO: `/v3/projects/3997d3abea6b443b9df2e09c7ad2092e` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:07.677132 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb4912f-a88b-4816-bc7d-d91a8bf1c1da None admin] RBAC: Authorizing `identity:get_project(project_id=3997d3abea6b443b9df2e09c7ad2092e)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:07.678616 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb4912f-a88b-4816-bc7d-d91a8bf1c1da None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:07.680352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 554/1129] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:29:07 2022] GET /identity/v3/projects/3997d3abea6b443b9df2e09c7ad2092e => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:07.686510 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c6fa29-d5a5-4979-91e3-97e11ea72832 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:07.778450 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:07.784710 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c6fa29-d5a5-4979-91e3-97e11ea72832 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:07.818113 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c23420-e80a-4210-973a-761b98023986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:07.818756 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c23420-e80a-4210-973a-761b98023986 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:07.819043 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c23420-e80a-4210-973a-761b98023986 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:07.819323 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c23420-e80a-4210-973a-761b98023986 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:07.820154 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c23420-e80a-4210-973a-761b98023986 None admin] RBAC: Adding query filter params (role.id=d6ada62a74a94222b864a6c283f8d9b1, scope.project.id=3997d3abea6b443b9df2e09c7ad2092e, user.id=b51e0e5be73a4c7aac122a2e66d5a326) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:07.820469 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c23420-e80a-4210-973a-761b98023986 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:07.822767 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c23420-e80a-4210-973a-761b98023986 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:07.828698 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 576/1130] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Jun 24 08:29:07 2022] GET /identity/v3/role_assignments?user.id=b51e0e5be73a4c7aac122a2e66d5a326&scope.project.id=3997d3abea6b443b9df2e09c7ad2092e&role.id=d6ada62a74a94222b864a6c283f8d9b1 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:09.858790 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36192fc-4fdc-47c9-b2af-503ca32bc4b3 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:09.859319 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36192fc-4fdc-47c9-b2af-503ca32bc4b3 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:09.859319 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36192fc-4fdc-47c9-b2af-503ca32bc4b3 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:09.860695 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 555/1131] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:29:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:09.865785 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a13b4-2015-49cd-9be1-211af42cf411 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:09.866210 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a13b4-2015-49cd-9be1-211af42cf411 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:09.866349 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a13b4-2015-49cd-9be1-211af42cf411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:09.884002 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-846a13b4-2015-49cd-9be1-211af42cf411 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:09.885603 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-846a13b4-2015-49cd-9be1-211af42cf411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:09.952188 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 577/1132] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:29:09.983150 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36192fc-4fdc-47c9-b2af-503ca32bc4b3 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:10.084744 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36192fc-4fdc-47c9-b2af-503ca32bc4b3 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:10.119402 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72f3658-36e8-433c-8076-6b7f42980ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:10.120118 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f3658-36e8-433c-8076-6b7f42980ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:10.120402 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f3658-36e8-433c-8076-6b7f42980ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:10.120677 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f3658-36e8-433c-8076-6b7f42980ad8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:10.121863 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72f3658-36e8-433c-8076-6b7f42980ad8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:10.123301 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72f3658-36e8-433c-8076-6b7f42980ad8 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:10.124903 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 556/1133] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:10.130984 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846a13b4-2015-49cd-9be1-211af42cf411 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:10.222545 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:10.233065 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846a13b4-2015-49cd-9be1-211af42cf411 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:10.267066 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecfef04-d37a-4e69-8ae3-6be4784ed05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:10.267711 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecfef04-d37a-4e69-8ae3-6be4784ed05c None admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:10.267992 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecfef04-d37a-4e69-8ae3-6be4784ed05c None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:10.268265 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecfef04-d37a-4e69-8ae3-6be4784ed05c None admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:10.268971 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecfef04-d37a-4e69-8ae3-6be4784ed05c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:10.270417 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecfef04-d37a-4e69-8ae3-6be4784ed05c None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:10.299494 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 578/1134] 10.222.0.13 () {60 vars in 1141 bytes} [Fri Jun 24 08:29:10 2022] POST /identity/v3/projects => generated 311 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:29:12.350365 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600edd19-bef7-48de-9aeb-3a42247c7e69 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:12.350922 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600edd19-bef7-48de-9aeb-3a42247c7e69 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:12.350922 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600edd19-bef7-48de-9aeb-3a42247c7e69 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:12.352375 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 557/1135] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:29:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:12.357820 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba8282-8dcb-4617-8d47-646125054eb5 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:12.358264 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba8282-8dcb-4617-8d47-646125054eb5 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:12.358356 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba8282-8dcb-4617-8d47-646125054eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:12.375649 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-00ba8282-8dcb-4617-8d47-646125054eb5 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:12.376848 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-00ba8282-8dcb-4617-8d47-646125054eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:12.443483 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 579/1136] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:29:12.469869 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600edd19-bef7-48de-9aeb-3a42247c7e69 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:12.519181 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:12.568490 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600edd19-bef7-48de-9aeb-3a42247c7e69 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:12.601062 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63418d4-40ed-46ce-8112-fd466bfb1f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:12.601797 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63418d4-40ed-46ce-8112-fd466bfb1f87 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:12.602080 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63418d4-40ed-46ce-8112-fd466bfb1f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:12.602387 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63418d4-40ed-46ce-8112-fd466bfb1f87 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:12.603539 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63418d4-40ed-46ce-8112-fd466bfb1f87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:12.604961 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63418d4-40ed-46ce-8112-fd466bfb1f87 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:12.606853 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 558/1137] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:12.612536 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ba8282-8dcb-4617-8d47-646125054eb5 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:12.682330 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:12.712436 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ba8282-8dcb-4617-8d47-646125054eb5 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:12.746281 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3058c50-9ff1-4e0e-8091-46d5094d5234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:12.746934 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3058c50-9ff1-4e0e-8091-46d5094d5234 None admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:12.747225 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3058c50-9ff1-4e0e-8091-46d5094d5234 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:12.747505 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3058c50-9ff1-4e0e-8091-46d5094d5234 None admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:12.748203 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3058c50-9ff1-4e0e-8091-46d5094d5234 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:12.749620 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3058c50-9ff1-4e0e-8091-46d5094d5234 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:12.769056 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 580/1138] 10.222.0.13 () {60 vars in 1132 bytes} [Fri Jun 24 08:29:12 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:29:14.869526 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8bcf0-b767-4b25-9953-974d0ee040d7 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:14.870104 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8bcf0-b767-4b25-9953-974d0ee040d7 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:14.870104 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8bcf0-b767-4b25-9953-974d0ee040d7 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:14.871598 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 559/1139] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:29:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:14.876815 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236b5966-8a9c-423a-8b21-95f349741f5c None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:14.877210 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236b5966-8a9c-423a-8b21-95f349741f5c None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:14.877377 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236b5966-8a9c-423a-8b21-95f349741f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:14.894690 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-236b5966-8a9c-423a-8b21-95f349741f5c None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:14.895863 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-236b5966-8a9c-423a-8b21-95f349741f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:14.961623 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 581/1140] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:29:14.985970 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf8bcf0-b767-4b25-9953-974d0ee040d7 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:15.085365 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf8bcf0-b767-4b25-9953-974d0ee040d7 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:15.101406 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:15.120792 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b9c0c8-73cd-46c1-8856-86320c6f6691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:15.121489 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9c0c8-73cd-46c1-8856-86320c6f6691 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:15.121811 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9c0c8-73cd-46c1-8856-86320c6f6691 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:15.122091 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9c0c8-73cd-46c1-8856-86320c6f6691 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:15.132169 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b9c0c8-73cd-46c1-8856-86320c6f6691 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:29:15.140037 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-77b9c0c8-73cd-46c1-8856-86320c6f6691 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:29:15.140037 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:15.140037 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:15.140037 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:15.140037 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:15.140037 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:15.140037 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:15.140037 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:15.140037 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:15.140037 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:15.140037 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:15.140037 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:15.140037 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:29:15.140037 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:29:15.140037 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:29:15.140037 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:29:15.141220 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:29:15.141220 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:29:15.141220 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:29:15.141220 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:29:15.141220 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:15.141220 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:15.141220 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:15.141220 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:15.141220 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:15.141220 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:15.141220 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:15.141220 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:15.141220 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:15.141220 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:15.141220 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:15.141220 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:15.142362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:15.142362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:15.142362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:15.142362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:15.142362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:29:15.142362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:29:15.142362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:29:15.142362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:29:15.142362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:29:15.142362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:29:15.142362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:29:15.142362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:29:15.142362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 560/1141] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Jun 24 08:29:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:15.147305 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236b5966-8a9c-423a-8b21-95f349741f5c None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:15.204182 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:15.246253 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236b5966-8a9c-423a-8b21-95f349741f5c None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:15.279145 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d754d4-3eda-4a4f-9514-bfd4dd55bbfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:15.279803 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d754d4-3eda-4a4f-9514-bfd4dd55bbfa None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:15.280089 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d754d4-3eda-4a4f-9514-bfd4dd55bbfa None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:15.280367 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d754d4-3eda-4a4f-9514-bfd4dd55bbfa None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:15.281114 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d754d4-3eda-4a4f-9514-bfd4dd55bbfa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:15.281423 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d754d4-3eda-4a4f-9514-bfd4dd55bbfa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:15.282902 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d754d4-3eda-4a4f-9514-bfd4dd55bbfa None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:15.291364 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 582/1142] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:29:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:15.297237 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b9c0c8-73cd-46c1-8856-86320c6f6691 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:15.398369 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b9c0c8-73cd-46c1-8856-86320c6f6691 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:15.433318 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d91158-e00e-4a33-b9b6-cbcd04fae8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:15.434207 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d91158-e00e-4a33-b9b6-cbcd04fae8bf None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:15.434494 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d91158-e00e-4a33-b9b6-cbcd04fae8bf None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:15.434768 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d91158-e00e-4a33-b9b6-cbcd04fae8bf None admin] PATH_INFO: `/v3/users/be4890a5128e4af7b8f54a4a04899c03` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:15.447744 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d91158-e00e-4a33-b9b6-cbcd04fae8bf None admin] RBAC: Authorizing `identity:get_user(user_id=be4890a5128e4af7b8f54a4a04899c03)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:15.449190 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d91158-e00e-4a33-b9b6-cbcd04fae8bf None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:15.450881 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 561/1143] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Jun 24 08:29:15 2022] GET /identity/v3/users/be4890a5128e4af7b8f54a4a04899c03 => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:15.456660 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d754d4-3eda-4a4f-9514-bfd4dd55bbfa None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:15.557708 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d754d4-3eda-4a4f-9514-bfd4dd55bbfa None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:15.577291 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:15.591153 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92c3bef-d974-456e-a679-56e436a9564b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:15.591793 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c3bef-d974-456e-a679-56e436a9564b None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:15.592079 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c3bef-d974-456e-a679-56e436a9564b None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:15.592357 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c3bef-d974-456e-a679-56e436a9564b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:15.658756 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92c3bef-d974-456e-a679-56e436a9564b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:15.660134 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92c3bef-d974-456e-a679-56e436a9564b None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:15.755562 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 583/1144] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:29:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 08:29:15.762125 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d91158-e00e-4a33-b9b6-cbcd04fae8bf None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:15.795326 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:15.860068 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d91158-e00e-4a33-b9b6-cbcd04fae8bf None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:15.892856 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c1c5d-341d-4ee5-ba2d-d7192014aff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:15.893712 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c1c5d-341d-4ee5-ba2d-d7192014aff4 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:15.894046 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c1c5d-341d-4ee5-ba2d-d7192014aff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:15.894287 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c1c5d-341d-4ee5-ba2d-d7192014aff4 None admin] PATH_INFO: `/v3/projects/4fd0a6f2e26840e68cfff5703a218c71` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:15.895438 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2c1c5d-341d-4ee5-ba2d-d7192014aff4 None admin] RBAC: Authorizing `identity:get_project(project_id=4fd0a6f2e26840e68cfff5703a218c71)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:15.896864 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2c1c5d-341d-4ee5-ba2d-d7192014aff4 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:15.898588 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 562/1145] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:29:15 2022] GET /identity/v3/projects/4fd0a6f2e26840e68cfff5703a218c71 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:15.905235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92c3bef-d974-456e-a679-56e436a9564b None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:15.925873 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:16.002040 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92c3bef-d974-456e-a679-56e436a9564b None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:16.035804 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc77096-4902-485e-9aab-e9c77f7a8d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:16.036446 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc77096-4902-485e-9aab-e9c77f7a8d62 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:16.036738 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc77096-4902-485e-9aab-e9c77f7a8d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:16.037015 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc77096-4902-485e-9aab-e9c77f7a8d62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:16.037862 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc77096-4902-485e-9aab-e9c77f7a8d62 None admin] RBAC: Adding query filter params (role.id=acfe707eee6944ad94d1915fc9ee7ea3, scope.project.id=4fd0a6f2e26840e68cfff5703a218c71, user.id=be4890a5128e4af7b8f54a4a04899c03) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:16.038176 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc77096-4902-485e-9aab-e9c77f7a8d62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:16.039596 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc77096-4902-485e-9aab-e9c77f7a8d62 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:16.045516 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 584/1146] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Jun 24 08:29:15 2022] GET /identity/v3/role_assignments?user.id=be4890a5128e4af7b8f54a4a04899c03&scope.project.id=4fd0a6f2e26840e68cfff5703a218c71&role.id=acfe707eee6944ad94d1915fc9ee7ea3 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:18.062471 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e2a01-22f2-4447-8a9a-2868aeaef7aa None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:18.062995 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e2a01-22f2-4447-8a9a-2868aeaef7aa None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:18.062995 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e2a01-22f2-4447-8a9a-2868aeaef7aa None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:18.064373 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 563/1147] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:29:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:18.069431 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb7d09f-3c7c-4d75-94af-68d13335b2af None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:18.069818 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb7d09f-3c7c-4d75-94af-68d13335b2af None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:18.070030 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb7d09f-3c7c-4d75-94af-68d13335b2af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:18.093199 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-2cb7d09f-3c7c-4d75-94af-68d13335b2af None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:18.094442 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-2cb7d09f-3c7c-4d75-94af-68d13335b2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:18.163249 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 585/1148] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:29:18.185671 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776e2a01-22f2-4447-8a9a-2868aeaef7aa None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:18.286591 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776e2a01-22f2-4447-8a9a-2868aeaef7aa None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:18.288050 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:18.320337 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96aefa2-b571-425a-8d68-93e3a2abd572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:18.321648 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96aefa2-b571-425a-8d68-93e3a2abd572 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:18.321648 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96aefa2-b571-425a-8d68-93e3a2abd572 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:18.322316 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96aefa2-b571-425a-8d68-93e3a2abd572 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:18.331959 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96aefa2-b571-425a-8d68-93e3a2abd572 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:29:18.340243 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-b96aefa2-b571-425a-8d68-93e3a2abd572 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:29:18.340243 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:18.340243 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:18.340243 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:18.340243 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:18.340243 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:18.340243 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:18.340243 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:18.340243 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:18.340243 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:18.340243 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:18.340243 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:18.340243 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:29:18.340243 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:29:18.340243 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:29:18.340243 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:29:18.341286 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:29:18.341286 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:29:18.341286 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:29:18.341286 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:29:18.341286 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:18.341286 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:18.341286 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:18.341286 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:18.341286 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:18.341286 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:18.341286 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:18.341286 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:18.341286 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:18.341286 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:18.341286 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:18.341286 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:18.342148 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:18.342148 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:18.342148 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:18.342148 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:18.342148 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:29:18.342148 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:29:18.342148 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:29:18.342148 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:29:18.342148 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:29:18.342148 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:29:18.342148 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:29:18.342148 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:29:18.342148 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 564/1149] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Jun 24 08:29:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:18.347734 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb7d09f-3c7c-4d75-94af-68d13335b2af None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:18.363684 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:18.449546 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb7d09f-3c7c-4d75-94af-68d13335b2af None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:18.482784 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61cb053-28df-44ae-8349-2f987d4942bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:18.483452 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61cb053-28df-44ae-8349-2f987d4942bf None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:18.483739 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61cb053-28df-44ae-8349-2f987d4942bf None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:18.484019 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61cb053-28df-44ae-8349-2f987d4942bf None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:18.484765 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61cb053-28df-44ae-8349-2f987d4942bf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:18.485075 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61cb053-28df-44ae-8349-2f987d4942bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:18.486468 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61cb053-28df-44ae-8349-2f987d4942bf None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:18.495214 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 586/1150] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:29:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:18.500951 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96aefa2-b571-425a-8d68-93e3a2abd572 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:18.599511 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96aefa2-b571-425a-8d68-93e3a2abd572 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:18.632518 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839e41b6-5b24-492c-9508-ac2dd20f511b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:18.633363 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e41b6-5b24-492c-9508-ac2dd20f511b None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:18.634328 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e41b6-5b24-492c-9508-ac2dd20f511b None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:18.634603 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e41b6-5b24-492c-9508-ac2dd20f511b None admin] PATH_INFO: `/v3/users/be4890a5128e4af7b8f54a4a04899c03` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:18.635762 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839e41b6-5b24-492c-9508-ac2dd20f511b None admin] RBAC: Authorizing `identity:get_user(user_id=be4890a5128e4af7b8f54a4a04899c03)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:18.637220 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839e41b6-5b24-492c-9508-ac2dd20f511b None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:18.638855 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 565/1151] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Jun 24 08:29:18 2022] GET /identity/v3/users/be4890a5128e4af7b8f54a4a04899c03 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:18.644497 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61cb053-28df-44ae-8349-2f987d4942bf None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:18.718489 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:18.740855 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61cb053-28df-44ae-8349-2f987d4942bf None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:18.774080 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eda4b1a-e594-4efd-9225-670be604da42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:18.774935 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda4b1a-e594-4efd-9225-670be604da42 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:18.775246 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda4b1a-e594-4efd-9225-670be604da42 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:18.775523 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda4b1a-e594-4efd-9225-670be604da42 None admin] PATH_INFO: `/v3/projects/4fd0a6f2e26840e68cfff5703a218c71` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:18.776742 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eda4b1a-e594-4efd-9225-670be604da42 None admin] RBAC: Authorizing `identity:get_project(project_id=4fd0a6f2e26840e68cfff5703a218c71)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:18.778234 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eda4b1a-e594-4efd-9225-670be604da42 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:18.779928 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 587/1152] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:29:18 2022] GET /identity/v3/projects/4fd0a6f2e26840e68cfff5703a218c71 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:29:18.785676 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839e41b6-5b24-492c-9508-ac2dd20f511b None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:18.816854 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:18.886115 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839e41b6-5b24-492c-9508-ac2dd20f511b None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:18.918860 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d604be07-1ec5-42c3-a695-ea16a0a578d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:18.919729 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604be07-1ec5-42c3-a695-ea16a0a578d9 None admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:18.920012 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604be07-1ec5-42c3-a695-ea16a0a578d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:18.920286 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604be07-1ec5-42c3-a695-ea16a0a578d9 None admin] PATH_INFO: `/v3/projects/4fd0a6f2e26840e68cfff5703a218c71/users/be4890a5128e4af7b8f54a4a04899c03/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:18.922264 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d604be07-1ec5-42c3-a695-ea16a0a578d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=4fd0a6f2e26840e68cfff5703a218c71, user_id=be4890a5128e4af7b8f54a4a04899c03, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:18.923695 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d604be07-1ec5-42c3-a695-ea16a0a578d9 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:18.936750 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 566/1153] 10.222.0.13 () {58 vars in 1439 bytes} [Fri Jun 24 08:29:18 2022] PUT /identity/v3/projects/4fd0a6f2e26840e68cfff5703a218c71/users/be4890a5128e4af7b8f54a4a04899c03/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:29:20.956655 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7bf317-d689-4b2e-8734-8e966d2afe4f None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:20.957197 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7bf317-d689-4b2e-8734-8e966d2afe4f None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:20.957197 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7bf317-d689-4b2e-8734-8e966d2afe4f None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:20.958735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 588/1154] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:29:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:20.964427 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638dcfcc-76de-483f-9dbe-5adabb7a6388 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:20.964893 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638dcfcc-76de-483f-9dbe-5adabb7a6388 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:20.964987 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638dcfcc-76de-483f-9dbe-5adabb7a6388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:20.982312 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-638dcfcc-76de-483f-9dbe-5adabb7a6388 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:20.983481 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-638dcfcc-76de-483f-9dbe-5adabb7a6388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:21.049395 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 567/1155] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:29:21.073811 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7bf317-d689-4b2e-8734-8e966d2afe4f None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:21.174288 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7bf317-d689-4b2e-8734-8e966d2afe4f None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:21.206400 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfa4b2c-74b1-4393-80df-8f6ce2858bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:21.207219 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa4b2c-74b1-4393-80df-8f6ce2858bce None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:21.207451 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa4b2c-74b1-4393-80df-8f6ce2858bce None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:21.207727 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa4b2c-74b1-4393-80df-8f6ce2858bce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:21.218108 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfa4b2c-74b1-4393-80df-8f6ce2858bce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:29:21.225508 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-ccfa4b2c-74b1-4393-80df-8f6ce2858bce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:29:21.225508 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:21.225508 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:21.225508 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:21.225508 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:21.225508 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:21.225508 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:21.225508 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:21.225508 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:21.225508 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:21.225508 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:21.225508 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:21.225508 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:29:21.225508 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:29:21.225508 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:29:21.225508 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:29:21.226641 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:29:21.226641 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:29:21.226641 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:29:21.226641 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:29:21.226641 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:21.226641 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:21.226641 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:21.226641 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:21.226641 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:21.226641 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:21.226641 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:21.226641 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:21.226641 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:21.226641 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:21.226641 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:21.226641 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:21.228579 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:21.228579 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:21.228579 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:21.228579 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:21.228579 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:29:21.228579 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:29:21.228579 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:29:21.228579 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:29:21.228579 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:29:21.228579 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:29:21.228579 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:29:21.228579 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:29:21.228579 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 589/1156] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Jun 24 08:29:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:21.233256 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638dcfcc-76de-483f-9dbe-5adabb7a6388 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:21.255193 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:21.330011 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638dcfcc-76de-483f-9dbe-5adabb7a6388 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:21.363602 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e00f52-b4c7-4e6d-9dbe-a251ccbd1090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:21.364263 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e00f52-b4c7-4e6d-9dbe-a251ccbd1090 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:21.364544 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e00f52-b4c7-4e6d-9dbe-a251ccbd1090 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:21.364817 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e00f52-b4c7-4e6d-9dbe-a251ccbd1090 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:21.365555 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e00f52-b4c7-4e6d-9dbe-a251ccbd1090 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:21.365896 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e00f52-b4c7-4e6d-9dbe-a251ccbd1090 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:21.367272 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e00f52-b4c7-4e6d-9dbe-a251ccbd1090 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:21.375585 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 568/1157] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:29:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:21.381515 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfa4b2c-74b1-4393-80df-8f6ce2858bce None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:21.385360 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:21.482749 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfa4b2c-74b1-4393-80df-8f6ce2858bce None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:21.515052 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d55042-0d1a-40d0-9c90-29d464daf962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:21.515902 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d55042-0d1a-40d0-9c90-29d464daf962 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:21.516182 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d55042-0d1a-40d0-9c90-29d464daf962 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:21.516454 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d55042-0d1a-40d0-9c90-29d464daf962 None admin] PATH_INFO: `/v3/users/be4890a5128e4af7b8f54a4a04899c03` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:21.517613 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d55042-0d1a-40d0-9c90-29d464daf962 None admin] RBAC: Authorizing `identity:get_user(user_id=be4890a5128e4af7b8f54a4a04899c03)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:21.519054 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d55042-0d1a-40d0-9c90-29d464daf962 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:21.520665 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 590/1158] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Jun 24 08:29:21 2022] GET /identity/v3/users/be4890a5128e4af7b8f54a4a04899c03 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:21.526883 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e00f52-b4c7-4e6d-9dbe-a251ccbd1090 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:21.606907 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:21.624692 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e00f52-b4c7-4e6d-9dbe-a251ccbd1090 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:21.657451 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d43aa-f005-4ec9-a690-2bd43765270f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:21.658121 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d43aa-f005-4ec9-a690-2bd43765270f None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:21.658409 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d43aa-f005-4ec9-a690-2bd43765270f None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:21.658690 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d43aa-f005-4ec9-a690-2bd43765270f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:21.724869 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d43aa-f005-4ec9-a690-2bd43765270f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:21.726351 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d43aa-f005-4ec9-a690-2bd43765270f None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:21.803954 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:21.823154 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 569/1159] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:29:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 08:29:21.829049 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d55042-0d1a-40d0-9c90-29d464daf962 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:21.901156 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:21.930972 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d55042-0d1a-40d0-9c90-29d464daf962 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:21.965280 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5724274d-b203-4c4e-81d2-68cce55ed2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:21.966164 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5724274d-b203-4c4e-81d2-68cce55ed2e9 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:21.966448 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5724274d-b203-4c4e-81d2-68cce55ed2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:21.966725 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5724274d-b203-4c4e-81d2-68cce55ed2e9 None admin] PATH_INFO: `/v3/projects/4fd0a6f2e26840e68cfff5703a218c71` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:21.967905 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5724274d-b203-4c4e-81d2-68cce55ed2e9 None admin] RBAC: Authorizing `identity:get_project(project_id=4fd0a6f2e26840e68cfff5703a218c71)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:21.969341 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5724274d-b203-4c4e-81d2-68cce55ed2e9 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:21.971063 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 591/1160] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:29:21 2022] GET /identity/v3/projects/4fd0a6f2e26840e68cfff5703a218c71 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:21.977077 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d43aa-f005-4ec9-a690-2bd43765270f None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:22.074245 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d43aa-f005-4ec9-a690-2bd43765270f None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:22.108690 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd428c7-2a3e-4468-83c8-cc458a66b44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:22.109327 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd428c7-2a3e-4468-83c8-cc458a66b44c None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:22.109607 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd428c7-2a3e-4468-83c8-cc458a66b44c None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:22.109905 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd428c7-2a3e-4468-83c8-cc458a66b44c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:22.110714 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd428c7-2a3e-4468-83c8-cc458a66b44c None admin] RBAC: Adding query filter params (role.id=acfe707eee6944ad94d1915fc9ee7ea3, scope.project.id=4fd0a6f2e26840e68cfff5703a218c71, user.id=be4890a5128e4af7b8f54a4a04899c03) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:22.111027 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd428c7-2a3e-4468-83c8-cc458a66b44c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:22.112444 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd428c7-2a3e-4468-83c8-cc458a66b44c None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:22.118749 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 570/1161] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Jun 24 08:29:21 2022] GET /identity/v3/role_assignments?user.id=be4890a5128e4af7b8f54a4a04899c03&scope.project.id=4fd0a6f2e26840e68cfff5703a218c71&role.id=acfe707eee6944ad94d1915fc9ee7ea3 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:24.195771 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f455f1-b931-4b64-b52b-2a0c9836bd09 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:24.196332 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f455f1-b931-4b64-b52b-2a0c9836bd09 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:24.196332 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f455f1-b931-4b64-b52b-2a0c9836bd09 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:24.197718 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 592/1162] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:29:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:24.206828 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb67ed0-275b-42ec-9127-5516fc99a4b1 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:24.207250 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb67ed0-275b-42ec-9127-5516fc99a4b1 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:24.207438 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb67ed0-275b-42ec-9127-5516fc99a4b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:24.223893 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-eeb67ed0-275b-42ec-9127-5516fc99a4b1 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:24.225079 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-eeb67ed0-275b-42ec-9127-5516fc99a4b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:24.267932 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:24.292574 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 571/1163] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:29:24.315524 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f455f1-b931-4b64-b52b-2a0c9836bd09 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:24.416629 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f455f1-b931-4b64-b52b-2a0c9836bd09 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:24.452558 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc0d46-3b35-4e37-bf94-693c10d8d1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:24.453177 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc0d46-3b35-4e37-bf94-693c10d8d1f8 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:24.453471 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc0d46-3b35-4e37-bf94-693c10d8d1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:24.453795 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc0d46-3b35-4e37-bf94-693c10d8d1f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:24.465603 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:24.523365 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bc0d46-3b35-4e37-bf94-693c10d8d1f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:24.525426 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bc0d46-3b35-4e37-bf94-693c10d8d1f8 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:24.620099 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 593/1164] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:29:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 08:29:24.626570 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb67ed0-275b-42ec-9127-5516fc99a4b1 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:24.721436 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb67ed0-275b-42ec-9127-5516fc99a4b1 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:24.755795 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5311cf29-7762-4b0b-9d9a-c2b4b93a5895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:24.756584 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311cf29-7762-4b0b-9d9a-c2b4b93a5895 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:24.756873 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311cf29-7762-4b0b-9d9a-c2b4b93a5895 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:24.757146 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311cf29-7762-4b0b-9d9a-c2b4b93a5895 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:24.773921 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5311cf29-7762-4b0b-9d9a-c2b4b93a5895 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:24.775367 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5311cf29-7762-4b0b-9d9a-c2b4b93a5895 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:24.794180 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-5311cf29-7762-4b0b-9d9a-c2b4b93a5895 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 24 08:29:24.794180 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:24.794180 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 24 08:29:24.794180 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 24 08:29:24.794180 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:29:24.794180 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:29:24.794180 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:29:24.794180 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:29:24.794180 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 24 08:29:24.794180 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:29:24.794180 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:29:24.794180 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:29:24.794180 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:24.794180 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:24.794180 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:24.794180 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:24.794180 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:24.795406 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:24.795406 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:24.795406 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:24.795406 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:24.795406 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:24.795406 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:24.795406 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:24.795406 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:24.795406 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:24.795406 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:24.795406 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:24.795406 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:24.795406 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:24.795406 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:24.795406 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:24.795406 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:24.796311 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:24.796311 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:24.796311 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:24.796311 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:24.796311 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:24.796311 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:24.796311 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:24.796311 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:24.796311 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:24.796311 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:24.796311 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 24 08:29:24.796311 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:24.796311 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 24 08:29:24.796311 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:29:24.796311 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 572/1165] 10.222.0.13 () {56 vars in 1117 bytes} [Fri Jun 24 08:29:24 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:24.801294 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc0d46-3b35-4e37-bf94-693c10d8d1f8 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:24.851121 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:24.907964 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc0d46-3b35-4e37-bf94-693c10d8d1f8 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:24.940819 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2c0a25-dad1-493f-a6e8-255a9d67f439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:24.941460 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c0a25-dad1-493f-a6e8-255a9d67f439 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:24.941767 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c0a25-dad1-493f-a6e8-255a9d67f439 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:24.942036 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c0a25-dad1-493f-a6e8-255a9d67f439 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:24.942738 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2c0a25-dad1-493f-a6e8-255a9d67f439 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:24.943037 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2c0a25-dad1-493f-a6e8-255a9d67f439 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:24.944375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2c0a25-dad1-493f-a6e8-255a9d67f439 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:24.951822 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 594/1166] 10.222.0.13 () {56 vars in 1115 bytes} [Fri Jun 24 08:29:24 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:27.035866 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f15cc-c5f6-44dc-817a-5db10bcb29df None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:27.036458 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f15cc-c5f6-44dc-817a-5db10bcb29df None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:27.036458 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f15cc-c5f6-44dc-817a-5db10bcb29df None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:27.037927 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 573/1167] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:29:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:27.043505 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3601a182-cbca-49c8-9dd7-831512674c48 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:27.043916 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3601a182-cbca-49c8-9dd7-831512674c48 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:27.044086 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3601a182-cbca-49c8-9dd7-831512674c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:27.060436 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-3601a182-cbca-49c8-9dd7-831512674c48 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:27.061609 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-3601a182-cbca-49c8-9dd7-831512674c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:27.137041 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 595/1168] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:29:27.160439 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25f15cc-c5f6-44dc-817a-5db10bcb29df None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:27.195902 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:27.262032 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25f15cc-c5f6-44dc-817a-5db10bcb29df None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:27.295747 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e90420-36a3-4056-bb43-536aef34b9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:27.296328 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e90420-36a3-4056-bb43-536aef34b9c3 None admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:27.296621 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e90420-36a3-4056-bb43-536aef34b9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:27.296933 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e90420-36a3-4056-bb43-536aef34b9c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:27.297671 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e90420-36a3-4056-bb43-536aef34b9c3 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:27.299067 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e90420-36a3-4056-bb43-536aef34b9c3 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:27.314456 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 574/1169] 10.222.0.13 () {60 vars in 1138 bytes} [Fri Jun 24 08:29:27 2022] POST /identity/v3/domains => generated 267 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:29:29.364623 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de4876-32d3-4b0c-83b3-d87e5631c80d None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:29.365203 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de4876-32d3-4b0c-83b3-d87e5631c80d None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:29.365203 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de4876-32d3-4b0c-83b3-d87e5631c80d None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:29.366575 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 596/1170] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:29:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:29.372016 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c482faa2-0fe9-48cb-9c23-3ae54282638b None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:29.372016 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c482faa2-0fe9-48cb-9c23-3ae54282638b None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:29.372473 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c482faa2-0fe9-48cb-9c23-3ae54282638b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:29.389181 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-c482faa2-0fe9-48cb-9c23-3ae54282638b None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:29.390384 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-c482faa2-0fe9-48cb-9c23-3ae54282638b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:29.454970 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 575/1171] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:29:29.480058 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4de4876-32d3-4b0c-83b3-d87e5631c80d None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:29.529522 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:29.595785 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4de4876-32d3-4b0c-83b3-d87e5631c80d None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:29.627995 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c9d888-fcc9-4489-9e58-12b609fae467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:29.628785 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c9d888-fcc9-4489-9e58-12b609fae467 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:29.629064 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c9d888-fcc9-4489-9e58-12b609fae467 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:29.629335 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c9d888-fcc9-4489-9e58-12b609fae467 None admin] PATH_INFO: `/v3/domains/6176dba39b214ea391cd4fe4c8da1f96` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:29.639793 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c9d888-fcc9-4489-9e58-12b609fae467 None admin] RBAC: Authorizing `identity:get_domain(domain_id=6176dba39b214ea391cd4fe4c8da1f96)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:29.641217 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c9d888-fcc9-4489-9e58-12b609fae467 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:29.642867 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 597/1172] 10.222.0.13 () {56 vars in 1183 bytes} [Fri Jun 24 08:29:29 2022] GET /identity/v3/domains/6176dba39b214ea391cd4fe4c8da1f96 => generated 267 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:29.648772 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c482faa2-0fe9-48cb-9c23-3ae54282638b None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:29.675831 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:29.747213 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c482faa2-0fe9-48cb-9c23-3ae54282638b None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:29.780046 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a47344-98e8-432e-9523-f387e9085997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:29.780697 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a47344-98e8-432e-9523-f387e9085997 None admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:29.780979 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a47344-98e8-432e-9523-f387e9085997 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:29.781254 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a47344-98e8-432e-9523-f387e9085997 None admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:29.781975 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a47344-98e8-432e-9523-f387e9085997 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:29.783412 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a47344-98e8-432e-9523-f387e9085997 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:29.812109 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 576/1173] 10.222.0.13 () {60 vars in 1141 bytes} [Fri Jun 24 08:29:29 2022] POST /identity/v3/projects => generated 361 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:29:31.881270 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf59491-6cc5-4e7e-b224-b2836bfe42bc None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:31.881848 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf59491-6cc5-4e7e-b224-b2836bfe42bc None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:31.881848 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf59491-6cc5-4e7e-b224-b2836bfe42bc None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:31.883197 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 598/1174] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:29:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:31.890744 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd503b-0540-4681-be74-e8f5488e99e5 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:31.891156 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd503b-0540-4681-be74-e8f5488e99e5 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:31.891468 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd503b-0540-4681-be74-e8f5488e99e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:31.908909 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-55dd503b-0540-4681-be74-e8f5488e99e5 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:31.910146 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-55dd503b-0540-4681-be74-e8f5488e99e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:31.975062 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 577/1175] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:29:32.000587 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf59491-6cc5-4e7e-b224-b2836bfe42bc None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:32.093309 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:32.101107 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf59491-6cc5-4e7e-b224-b2836bfe42bc None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:32.135055 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff1b1fe-b552-4550-9147-6d371364f106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:32.135837 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1b1fe-b552-4550-9147-6d371364f106 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:32.136161 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1b1fe-b552-4550-9147-6d371364f106 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:32.136472 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1b1fe-b552-4550-9147-6d371364f106 None admin] PATH_INFO: `/v3/domains/6176dba39b214ea391cd4fe4c8da1f96` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:32.137631 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff1b1fe-b552-4550-9147-6d371364f106 None admin] RBAC: Authorizing `identity:get_domain(domain_id=6176dba39b214ea391cd4fe4c8da1f96)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:32.139074 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff1b1fe-b552-4550-9147-6d371364f106 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:32.140659 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 599/1176] 10.222.0.13 () {56 vars in 1183 bytes} [Fri Jun 24 08:29:31 2022] GET /identity/v3/domains/6176dba39b214ea391cd4fe4c8da1f96 => generated 267 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:32.146442 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dd503b-0540-4681-be74-e8f5488e99e5 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:32.156530 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:32.246648 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dd503b-0540-4681-be74-e8f5488e99e5 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:32.280719 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2869b-cc5d-4f1c-b2bc-eb882ad8d76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:32.281384 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2869b-cc5d-4f1c-b2bc-eb882ad8d76f None admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:32.281668 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2869b-cc5d-4f1c-b2bc-eb882ad8d76f None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:32.281986 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2869b-cc5d-4f1c-b2bc-eb882ad8d76f None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:32.282692 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec2869b-cc5d-4f1c-b2bc-eb882ad8d76f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:32.284114 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec2869b-cc5d-4f1c-b2bc-eb882ad8d76f None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:32.304263 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 578/1177] 10.222.0.13 () {60 vars in 1132 bytes} [Fri Jun 24 08:29:32 2022] POST /identity/v3/users => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:29:34.374632 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f313af35-4285-4e12-8a29-e6c5a68dcf22 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:34.375035 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f313af35-4285-4e12-8a29-e6c5a68dcf22 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:34.375185 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f313af35-4285-4e12-8a29-e6c5a68dcf22 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:34.376542 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 600/1178] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:29:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:34.381865 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac378a-a792-4e40-838e-dadbcd12860e None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:34.382241 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac378a-a792-4e40-838e-dadbcd12860e None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:34.382418 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac378a-a792-4e40-838e-dadbcd12860e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:34.400195 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-0fac378a-a792-4e40-838e-dadbcd12860e None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:34.401362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-0fac378a-a792-4e40-838e-dadbcd12860e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:34.457511 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:34.467559 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 579/1179] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:29:34.491335 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f313af35-4285-4e12-8a29-e6c5a68dcf22 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:34.591637 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f313af35-4285-4e12-8a29-e6c5a68dcf22 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:34.625670 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9b9cc2-2658-41b5-b113-4879ada3e8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:34.626421 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b9cc2-2658-41b5-b113-4879ada3e8c9 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:34.626708 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b9cc2-2658-41b5-b113-4879ada3e8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:34.626985 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b9cc2-2658-41b5-b113-4879ada3e8c9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:34.637242 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9b9cc2-2658-41b5-b113-4879ada3e8c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:29:34.644898 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-ae9b9cc2-2658-41b5-b113-4879ada3e8c9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:29:34.644898 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:34.644898 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:34.644898 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:34.644898 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:34.644898 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:34.644898 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:34.644898 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:34.644898 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:34.644898 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:34.644898 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:34.644898 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:34.644898 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:29:34.644898 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:29:34.644898 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:29:34.644898 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:29:34.646128 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:29:34.646128 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:29:34.646128 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:29:34.646128 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:29:34.646128 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:34.646128 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:34.646128 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:34.646128 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:34.646128 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:34.646128 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:34.646128 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:34.646128 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:34.646128 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:34.646128 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:34.646128 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:34.646128 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:34.647102 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:34.647102 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:34.647102 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:34.647102 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:34.647102 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:29:34.647102 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:29:34.647102 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:29:34.647102 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:29:34.647102 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:29:34.647102 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:29:34.647102 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:29:34.647102 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:29:34.647102 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 601/1180] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Jun 24 08:29:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:34.652177 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac378a-a792-4e40-838e-dadbcd12860e None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:34.748903 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac378a-a792-4e40-838e-dadbcd12860e None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:34.781628 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e40cbe-e407-4beb-af0f-1e2d6a8895e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:34.782309 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e40cbe-e407-4beb-af0f-1e2d6a8895e1 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:34.782593 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e40cbe-e407-4beb-af0f-1e2d6a8895e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:34.782869 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e40cbe-e407-4beb-af0f-1e2d6a8895e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:34.783602 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e40cbe-e407-4beb-af0f-1e2d6a8895e1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:34.783907 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e40cbe-e407-4beb-af0f-1e2d6a8895e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:34.785273 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e40cbe-e407-4beb-af0f-1e2d6a8895e1 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:34.793658 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 580/1181] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:29:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:34.799289 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9b9cc2-2658-41b5-b113-4879ada3e8c9 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:34.824147 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:34.899008 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9b9cc2-2658-41b5-b113-4879ada3e8c9 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:34.932216 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63eb1a8-08a4-40d9-b6ed-e1f464cbd3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:34.933075 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63eb1a8-08a4-40d9-b6ed-e1f464cbd3c7 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:34.933355 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63eb1a8-08a4-40d9-b6ed-e1f464cbd3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:34.933629 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63eb1a8-08a4-40d9-b6ed-e1f464cbd3c7 None admin] PATH_INFO: `/v3/users/1f864abd8d5c45b8958968629abecab5` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:34.948870 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63eb1a8-08a4-40d9-b6ed-e1f464cbd3c7 None admin] RBAC: Authorizing `identity:get_user(user_id=1f864abd8d5c45b8958968629abecab5)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:34.950425 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63eb1a8-08a4-40d9-b6ed-e1f464cbd3c7 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:34.952109 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 602/1182] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Jun 24 08:29:34 2022] GET /identity/v3/users/1f864abd8d5c45b8958968629abecab5 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:34.957877 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e40cbe-e407-4beb-af0f-1e2d6a8895e1 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:35.012548 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:35.055688 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e40cbe-e407-4beb-af0f-1e2d6a8895e1 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:35.088757 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caafc2b-a1c5-4f95-a08f-fde0a8d40099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:35.089389 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caafc2b-a1c5-4f95-a08f-fde0a8d40099 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:35.089672 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caafc2b-a1c5-4f95-a08f-fde0a8d40099 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:35.089976 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caafc2b-a1c5-4f95-a08f-fde0a8d40099 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:35.155203 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caafc2b-a1c5-4f95-a08f-fde0a8d40099 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:35.156587 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caafc2b-a1c5-4f95-a08f-fde0a8d40099 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:35.208887 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:35.252125 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 581/1183] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:29:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 08:29:35.258205 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63eb1a8-08a4-40d9-b6ed-e1f464cbd3c7 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:35.338220 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:35.359156 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63eb1a8-08a4-40d9-b6ed-e1f464cbd3c7 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:35.392848 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6985fc7-6cd6-4708-9986-27fcc3f2fc67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:35.393702 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6985fc7-6cd6-4708-9986-27fcc3f2fc67 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:35.393992 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6985fc7-6cd6-4708-9986-27fcc3f2fc67 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:35.394261 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6985fc7-6cd6-4708-9986-27fcc3f2fc67 None admin] PATH_INFO: `/v3/projects/4ff15e22078f4d69a9e9b9d6d4f6e70a` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:35.395413 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6985fc7-6cd6-4708-9986-27fcc3f2fc67 None admin] RBAC: Authorizing `identity:get_project(project_id=4ff15e22078f4d69a9e9b9d6d4f6e70a)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:35.396825 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6985fc7-6cd6-4708-9986-27fcc3f2fc67 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:35.398534 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 603/1184] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:29:35 2022] GET /identity/v3/projects/4ff15e22078f4d69a9e9b9d6d4f6e70a => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:35.404844 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caafc2b-a1c5-4f95-a08f-fde0a8d40099 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:35.501144 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caafc2b-a1c5-4f95-a08f-fde0a8d40099 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:35.534611 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52b81af-2ff4-4a64-b639-e87c6165599b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:35.535246 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b81af-2ff4-4a64-b639-e87c6165599b None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:35.535529 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b81af-2ff4-4a64-b639-e87c6165599b None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:35.535806 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b81af-2ff4-4a64-b639-e87c6165599b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:35.536614 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52b81af-2ff4-4a64-b639-e87c6165599b None admin] RBAC: Adding query filter params (role.id=acfe707eee6944ad94d1915fc9ee7ea3, scope.project.id=4ff15e22078f4d69a9e9b9d6d4f6e70a, user.id=1f864abd8d5c45b8958968629abecab5) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:35.536932 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52b81af-2ff4-4a64-b639-e87c6165599b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:35.538392 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52b81af-2ff4-4a64-b639-e87c6165599b None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:35.544560 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 582/1185] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Jun 24 08:29:35 2022] GET /identity/v3/role_assignments?user.id=1f864abd8d5c45b8958968629abecab5&scope.project.id=4ff15e22078f4d69a9e9b9d6d4f6e70a&role.id=acfe707eee6944ad94d1915fc9ee7ea3 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:37.520840 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653da4a3-5444-40ce-bdfc-d559923fa72e None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:37.521385 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653da4a3-5444-40ce-bdfc-d559923fa72e None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:37.521385 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653da4a3-5444-40ce-bdfc-d559923fa72e None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:37.522754 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 604/1186] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:29:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:29:37.528553 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfed14de-9165-4a63-b34d-f141f5ca6667 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:37.528940 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfed14de-9165-4a63-b34d-f141f5ca6667 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:37.529114 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfed14de-9165-4a63-b34d-f141f5ca6667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:37.545486 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-dfed14de-9165-4a63-b34d-f141f5ca6667 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:37.546709 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-dfed14de-9165-4a63-b34d-f141f5ca6667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:37.560184 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:37.610503 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 583/1187] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:29:37.632916 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653da4a3-5444-40ce-bdfc-d559923fa72e None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:37.733202 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653da4a3-5444-40ce-bdfc-d559923fa72e None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:37.766307 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaeae58-143f-4f28-baff-59b3556992e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:37.767078 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaeae58-143f-4f28-baff-59b3556992e7 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:37.767358 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaeae58-143f-4f28-baff-59b3556992e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:37.767630 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaeae58-143f-4f28-baff-59b3556992e7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:37.778039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaeae58-143f-4f28-baff-59b3556992e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:29:37.785445 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-dbaeae58-143f-4f28-baff-59b3556992e7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:29:37.785445 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:37.785445 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:37.785445 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:37.785445 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:37.785445 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:37.785445 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:37.785445 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:37.785445 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:37.785445 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:37.785445 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:37.785445 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:37.785445 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:29:37.785445 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:29:37.785445 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:29:37.785445 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:29:37.786531 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:29:37.786531 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:29:37.786531 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:29:37.786531 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:29:37.786531 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:37.786531 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:37.786531 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:37.786531 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:37.786531 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:37.786531 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:37.786531 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:37.786531 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:37.786531 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:37.786531 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:37.786531 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:37.786531 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:37.787600 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:37.787600 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:37.787600 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:37.787600 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:37.787600 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:29:37.787600 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:29:37.787600 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:29:37.787600 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:29:37.787600 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:29:37.787600 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:29:37.787600 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:29:37.787600 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:29:37.787600 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 605/1188] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Jun 24 08:29:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:37.792800 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfed14de-9165-4a63-b34d-f141f5ca6667 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:37.888224 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfed14de-9165-4a63-b34d-f141f5ca6667 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:37.919964 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a429dc-73be-4b44-b86f-e8875b8a2fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:37.920619 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a429dc-73be-4b44-b86f-e8875b8a2fe9 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:37.920901 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a429dc-73be-4b44-b86f-e8875b8a2fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:37.921173 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a429dc-73be-4b44-b86f-e8875b8a2fe9 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:37.921957 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a429dc-73be-4b44-b86f-e8875b8a2fe9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:37.922259 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a429dc-73be-4b44-b86f-e8875b8a2fe9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:37.923619 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a429dc-73be-4b44-b86f-e8875b8a2fe9 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:37.931933 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 584/1189] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:29:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:37.937567 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaeae58-143f-4f28-baff-59b3556992e7 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:37.949419 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:38.036619 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaeae58-143f-4f28-baff-59b3556992e7 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:38.068677 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4eabc6-325d-4e4d-8865-f5ca1eb4d55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:38.069521 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4eabc6-325d-4e4d-8865-f5ca1eb4d55e None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:38.069839 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4eabc6-325d-4e4d-8865-f5ca1eb4d55e None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:38.070113 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4eabc6-325d-4e4d-8865-f5ca1eb4d55e None admin] PATH_INFO: `/v3/users/1f864abd8d5c45b8958968629abecab5` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:38.072115 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4eabc6-325d-4e4d-8865-f5ca1eb4d55e None admin] RBAC: Authorizing `identity:get_user(user_id=1f864abd8d5c45b8958968629abecab5)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:38.073533 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4eabc6-325d-4e4d-8865-f5ca1eb4d55e None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:38.075166 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 606/1190] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Jun 24 08:29:37 2022] GET /identity/v3/users/1f864abd8d5c45b8958968629abecab5 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:38.080877 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a429dc-73be-4b44-b86f-e8875b8a2fe9 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:38.096588 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:38.182012 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a429dc-73be-4b44-b86f-e8875b8a2fe9 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:38.216603 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0794ac-c272-4128-822d-69bffc715c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:38.217458 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0794ac-c272-4128-822d-69bffc715c3f None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:38.217770 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0794ac-c272-4128-822d-69bffc715c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:38.218050 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0794ac-c272-4128-822d-69bffc715c3f None admin] PATH_INFO: `/v3/projects/4ff15e22078f4d69a9e9b9d6d4f6e70a` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:38.219216 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0794ac-c272-4128-822d-69bffc715c3f None admin] RBAC: Authorizing `identity:get_project(project_id=4ff15e22078f4d69a9e9b9d6d4f6e70a)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:38.220662 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0794ac-c272-4128-822d-69bffc715c3f None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:38.222398 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 585/1191] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:29:38 2022] GET /identity/v3/projects/4ff15e22078f4d69a9e9b9d6d4f6e70a => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:38.231108 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4eabc6-325d-4e4d-8865-f5ca1eb4d55e None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:38.314591 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:38.337894 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4eabc6-325d-4e4d-8865-f5ca1eb4d55e None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:38.371275 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0389de-7981-4696-beda-c5b24e6e8d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:38.372485 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0389de-7981-4696-beda-c5b24e6e8d31 None admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:38.372485 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0389de-7981-4696-beda-c5b24e6e8d31 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:38.372680 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0389de-7981-4696-beda-c5b24e6e8d31 None admin] PATH_INFO: `/v3/projects/4ff15e22078f4d69a9e9b9d6d4f6e70a/users/1f864abd8d5c45b8958968629abecab5/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:38.374659 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0389de-7981-4696-beda-c5b24e6e8d31 None admin] RBAC: Authorizing `identity:create_grant(project_id=4ff15e22078f4d69a9e9b9d6d4f6e70a, user_id=1f864abd8d5c45b8958968629abecab5, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:38.376083 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0389de-7981-4696-beda-c5b24e6e8d31 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:38.389517 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 607/1192] 10.222.0.13 () {58 vars in 1439 bytes} [Fri Jun 24 08:29:38 2022] PUT /identity/v3/projects/4ff15e22078f4d69a9e9b9d6d4f6e70a/users/1f864abd8d5c45b8958968629abecab5/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:29:40.439286 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14545d-e8f8-4b5e-bfab-bb74c74b583c None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:40.439834 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14545d-e8f8-4b5e-bfab-bb74c74b583c None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:40.439834 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14545d-e8f8-4b5e-bfab-bb74c74b583c None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:40.441238 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 586/1193] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:29:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:40.446535 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88af789-6227-47cc-a314-8d9e1666ba38 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:40.446943 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88af789-6227-47cc-a314-8d9e1666ba38 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:40.447089 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88af789-6227-47cc-a314-8d9e1666ba38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:40.464151 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-c88af789-6227-47cc-a314-8d9e1666ba38 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:40.465320 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-c88af789-6227-47cc-a314-8d9e1666ba38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:40.531564 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 608/1194] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:29:40.559784 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc14545d-e8f8-4b5e-bfab-bb74c74b583c None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:40.645059 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:40.664575 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc14545d-e8f8-4b5e-bfab-bb74c74b583c None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:40.700390 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5ec03f-618a-4c25-b614-dda4c9ecfc0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:40.701164 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5ec03f-618a-4c25-b614-dda4c9ecfc0e None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:40.701444 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5ec03f-618a-4c25-b614-dda4c9ecfc0e None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:40.701733 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5ec03f-618a-4c25-b614-dda4c9ecfc0e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:40.711756 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5ec03f-618a-4c25-b614-dda4c9ecfc0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:29:40.719421 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-2f5ec03f-618a-4c25-b614-dda4c9ecfc0e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:29:40.719421 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:40.719421 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:40.719421 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:40.719421 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:40.719421 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:40.719421 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:40.719421 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:40.719421 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:40.719421 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:40.719421 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:40.719421 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:40.719421 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:29:40.719421 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:29:40.719421 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:29:40.719421 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:29:40.720489 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:29:40.720489 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:29:40.720489 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:29:40.720489 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:29:40.720489 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:40.720489 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:40.720489 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:40.720489 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:40.720489 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:40.720489 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:40.720489 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:40.720489 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:40.720489 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:40.720489 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:40.720489 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:40.720489 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:40.721532 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:40.721532 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:40.721532 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:40.721532 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:40.721532 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:29:40.721532 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:29:40.721532 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:29:40.721532 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:29:40.721532 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:29:40.721532 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:29:40.721532 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:29:40.721532 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:29:40.721532 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 587/1195] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Jun 24 08:29:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:40.726625 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88af789-6227-47cc-a314-8d9e1666ba38 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:40.790975 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:40.825508 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88af789-6227-47cc-a314-8d9e1666ba38 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:40.872330 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040fe127-9466-489d-8428-d5ddba962755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:40.873009 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040fe127-9466-489d-8428-d5ddba962755 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:40.873290 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040fe127-9466-489d-8428-d5ddba962755 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:40.873560 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040fe127-9466-489d-8428-d5ddba962755 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:40.874368 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040fe127-9466-489d-8428-d5ddba962755 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:40.874671 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040fe127-9466-489d-8428-d5ddba962755 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:40.876986 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040fe127-9466-489d-8428-d5ddba962755 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:40.886314 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 609/1196] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:29:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:40.892172 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5ec03f-618a-4c25-b614-dda4c9ecfc0e None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:40.991686 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5ec03f-618a-4c25-b614-dda4c9ecfc0e None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:41.023510 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23351272-c4b5-420e-85c2-5fb86b5efe3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:41.024355 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23351272-c4b5-420e-85c2-5fb86b5efe3e None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:41.024634 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23351272-c4b5-420e-85c2-5fb86b5efe3e None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:41.025424 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23351272-c4b5-420e-85c2-5fb86b5efe3e None admin] PATH_INFO: `/v3/users/1f864abd8d5c45b8958968629abecab5` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:41.026747 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23351272-c4b5-420e-85c2-5fb86b5efe3e None admin] RBAC: Authorizing `identity:get_user(user_id=1f864abd8d5c45b8958968629abecab5)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:41.028180 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23351272-c4b5-420e-85c2-5fb86b5efe3e None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:41.029813 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 588/1197] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Jun 24 08:29:40 2022] GET /identity/v3/users/1f864abd8d5c45b8958968629abecab5 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:41.035544 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040fe127-9466-489d-8428-d5ddba962755 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:41.134919 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040fe127-9466-489d-8428-d5ddba962755 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:41.160522 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:41.169171 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64be7daa-49cf-4daa-b179-24bdd759668a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:41.169836 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be7daa-49cf-4daa-b179-24bdd759668a None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:41.170120 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be7daa-49cf-4daa-b179-24bdd759668a None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:41.170390 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be7daa-49cf-4daa-b179-24bdd759668a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:41.237642 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64be7daa-49cf-4daa-b179-24bdd759668a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:41.239104 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64be7daa-49cf-4daa-b179-24bdd759668a None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:41.335370 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 610/1198] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:29:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 08:29:41.342001 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23351272-c4b5-420e-85c2-5fb86b5efe3e None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:41.353722 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:41.449594 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23351272-c4b5-420e-85c2-5fb86b5efe3e None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:41.482497 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039b4a6a-7573-452c-9c46-aebf1dec5924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:41.483350 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b4a6a-7573-452c-9c46-aebf1dec5924 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:41.483636 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b4a6a-7573-452c-9c46-aebf1dec5924 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:41.483910 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b4a6a-7573-452c-9c46-aebf1dec5924 None admin] PATH_INFO: `/v3/projects/4ff15e22078f4d69a9e9b9d6d4f6e70a` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:41.485075 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039b4a6a-7573-452c-9c46-aebf1dec5924 None admin] RBAC: Authorizing `identity:get_project(project_id=4ff15e22078f4d69a9e9b9d6d4f6e70a)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:41.486526 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039b4a6a-7573-452c-9c46-aebf1dec5924 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:41.488219 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 589/1199] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Jun 24 08:29:41 2022] GET /identity/v3/projects/4ff15e22078f4d69a9e9b9d6d4f6e70a => generated 361 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:41.494344 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64be7daa-49cf-4daa-b179-24bdd759668a None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:41.521704 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:41.591816 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64be7daa-49cf-4daa-b179-24bdd759668a None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:41.624376 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea21e6d-d9dd-46c3-91c4-53d5c2bd3493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:41.625009 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea21e6d-d9dd-46c3-91c4-53d5c2bd3493 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:41.625292 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea21e6d-d9dd-46c3-91c4-53d5c2bd3493 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:41.625563 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea21e6d-d9dd-46c3-91c4-53d5c2bd3493 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:41.626405 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea21e6d-d9dd-46c3-91c4-53d5c2bd3493 None admin] RBAC: Adding query filter params (role.id=acfe707eee6944ad94d1915fc9ee7ea3, scope.project.id=4ff15e22078f4d69a9e9b9d6d4f6e70a, user.id=1f864abd8d5c45b8958968629abecab5) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:41.626715 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea21e6d-d9dd-46c3-91c4-53d5c2bd3493 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:41.628129 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea21e6d-d9dd-46c3-91c4-53d5c2bd3493 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:41.634103 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 611/1200] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Jun 24 08:29:41 2022] GET /identity/v3/role_assignments?user.id=1f864abd8d5c45b8958968629abecab5&scope.project.id=4ff15e22078f4d69a9e9b9d6d4f6e70a&role.id=acfe707eee6944ad94d1915fc9ee7ea3 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:29:51.478343 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=62 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:51.631585 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=54 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:40.625358 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80448396-f08f-42e1-9246-bccdb4e7d59a None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:40.626090 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80448396-f08f-42e1-9246-bccdb4e7d59a None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:40.626207 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80448396-f08f-42e1-9246-bccdb4e7d59a None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:40.628822 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 590/1201] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:30:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:30:40.638575 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49098df3-29a9-461c-bda1-8ea52d0e3b83 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:40.639116 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49098df3-29a9-461c-bda1-8ea52d0e3b83 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:40.639463 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49098df3-29a9-461c-bda1-8ea52d0e3b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:40.663824 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-49098df3-29a9-461c-bda1-8ea52d0e3b83 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:30:40.665401 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-49098df3-29a9-461c-bda1-8ea52d0e3b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:30:40.751734 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 612/1202] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:30:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:30:40.794667 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80448396-f08f-42e1-9246-bccdb4e7d59a None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:41.013840 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80448396-f08f-42e1-9246-bccdb4e7d59a None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:41.070711 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffb47b3-231f-4fe1-9819-ccd969ea6e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:41.072525 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffb47b3-231f-4fe1-9819-ccd969ea6e0e None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:41.073035 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffb47b3-231f-4fe1-9819-ccd969ea6e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:41.073472 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffb47b3-231f-4fe1-9819-ccd969ea6e0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:41.090386 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffb47b3-231f-4fe1-9819-ccd969ea6e0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:41.091987 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffb47b3-231f-4fe1-9819-ccd969ea6e0e None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:41.122463 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-2ffb47b3-231f-4fe1-9819-ccd969ea6e0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:30:41.122463 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:30:41.122463 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:30:41.122463 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:30:41.122463 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:30:41.122463 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:30:41.122463 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:30:41.122463 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:30:41.122463 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:30:41.122463 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:30:41.122463 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:30:41.122463 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:30:41.122463 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:30:41.122463 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:30:41.122463 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:30:41.123686 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:30:41.123686 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:30:41.123686 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:30:41.123686 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:30:41.123686 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:30:41.123686 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:30:41.123686 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:30:41.123686 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:30:41.123686 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:30:41.123686 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:30:41.123686 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:30:41.123686 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:30:41.123686 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:30:41.123686 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:30:41.123686 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:30:41.123686 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:30:41.123686 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:30:41.124909 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:30:41.124909 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:30:41.124909 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:30:41.124909 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:30:41.131797 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 591/1203] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:30:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:30:41.133911 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49098df3-29a9-461c-bda1-8ea52d0e3b83 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:41.317576 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49098df3-29a9-461c-bda1-8ea52d0e3b83 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:41.367014 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:41.376149 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352b6c19-82f4-448f-b7a7-7079b753a7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:41.377063 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b6c19-82f4-448f-b7a7-7079b753a7f4 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:41.377732 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b6c19-82f4-448f-b7a7-7079b753a7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:41.377732 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b6c19-82f4-448f-b7a7-7079b753a7f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:41.379599 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352b6c19-82f4-448f-b7a7-7079b753a7f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:41.379599 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352b6c19-82f4-448f-b7a7-7079b753a7f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:41.380316 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352b6c19-82f4-448f-b7a7-7079b753a7f4 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:41.405967 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 613/1204] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:30:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:30:41.409741 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffb47b3-231f-4fe1-9819-ccd969ea6e0e None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:41.499420 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:41.606057 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffb47b3-231f-4fe1-9819-ccd969ea6e0e None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:41.678313 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde7758c-c7b2-4bd4-bb95-3af4cc0a16ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:41.679058 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde7758c-c7b2-4bd4-bb95-3af4cc0a16ec None admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:41.679308 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde7758c-c7b2-4bd4-bb95-3af4cc0a16ec None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:41.679574 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde7758c-c7b2-4bd4-bb95-3af4cc0a16ec None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:41.680323 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde7758c-c7b2-4bd4-bb95-3af4cc0a16ec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:41.682441 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde7758c-c7b2-4bd4-bb95-3af4cc0a16ec None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:41.713746 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 592/1205] 10.222.0.13 () {60 vars in 1132 bytes} [Fri Jun 24 08:30:41 2022] POST /identity/v3/users => generated 251 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:30:44.046191 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bbe599-2c48-41e8-b9e0-a311d0406178 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:44.047028 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bbe599-2c48-41e8-b9e0-a311d0406178 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:44.047028 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bbe599-2c48-41e8-b9e0-a311d0406178 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:44.048340 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 614/1206] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:30:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:30:44.056301 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306503e0-c4a1-481a-9a9a-c4b12dfe8d32 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:44.056924 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306503e0-c4a1-481a-9a9a-c4b12dfe8d32 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:44.057165 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306503e0-c4a1-481a-9a9a-c4b12dfe8d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:44.082415 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-306503e0-c4a1-481a-9a9a-c4b12dfe8d32 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:30:44.083731 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-306503e0-c4a1-481a-9a9a-c4b12dfe8d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:30:44.174295 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 593/1207] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:30:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:30:44.199424 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bbe599-2c48-41e8-b9e0-a311d0406178 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:44.377370 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bbe599-2c48-41e8-b9e0-a311d0406178 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:44.432638 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ae909-f13c-4b00-9bae-71868aea50d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:44.434333 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ae909-f13c-4b00-9bae-71868aea50d2 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:44.434961 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ae909-f13c-4b00-9bae-71868aea50d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:44.435552 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ae909-f13c-4b00-9bae-71868aea50d2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:44.454055 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327ae909-f13c-4b00-9bae-71868aea50d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:30:44.461837 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-327ae909-f13c-4b00-9bae-71868aea50d2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:30:44.461837 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:30:44.461837 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:30:44.461837 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:30:44.461837 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:30:44.461837 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:30:44.461837 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:30:44.461837 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:30:44.461837 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:30:44.461837 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:30:44.461837 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:30:44.461837 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:30:44.461837 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:30:44.461837 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:30:44.461837 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:30:44.461837 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:30:44.463999 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:30:44.463999 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:30:44.463999 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:30:44.463999 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:30:44.463999 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:30:44.463999 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:30:44.463999 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:30:44.463999 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:30:44.463999 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:30:44.463999 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:30:44.463999 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:30:44.463999 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:30:44.463999 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:30:44.463999 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:30:44.463999 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:30:44.463999 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:30:44.465682 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:30:44.465682 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:30:44.465682 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:30:44.465682 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:30:44.465682 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:30:44.465682 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:30:44.465682 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:30:44.465682 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:30:44.465682 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:30:44.465682 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:30:44.465682 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:30:44.465682 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:30:44.465682 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 615/1208] 10.222.0.13 () {56 vars in 1102 bytes} [Fri Jun 24 08:30:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:30:44.475348 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306503e0-c4a1-481a-9a9a-c4b12dfe8d32 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:44.531637 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:44.620766 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306503e0-c4a1-481a-9a9a-c4b12dfe8d32 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:44.669815 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53fa3b0-3f08-4e5c-882c-b0d844c721b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:44.669815 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53fa3b0-3f08-4e5c-882c-b0d844c721b3 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:44.669815 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53fa3b0-3f08-4e5c-882c-b0d844c721b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:44.669815 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53fa3b0-3f08-4e5c-882c-b0d844c721b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:44.670462 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53fa3b0-3f08-4e5c-882c-b0d844c721b3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:44.670743 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53fa3b0-3f08-4e5c-882c-b0d844c721b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:44.672076 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53fa3b0-3f08-4e5c-882c-b0d844c721b3 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:44.681903 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 594/1209] 10.222.0.13 () {56 vars in 1103 bytes} [Fri Jun 24 08:30:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:44.688798 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ae909-f13c-4b00-9bae-71868aea50d2 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:44.752423 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:44.817941 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ae909-f13c-4b00-9bae-71868aea50d2 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:44.857490 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d681d4ea-35dd-4707-b63f-78b5d638c1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:44.858860 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d681d4ea-35dd-4707-b63f-78b5d638c1b2 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:44.859209 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d681d4ea-35dd-4707-b63f-78b5d638c1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:44.859583 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d681d4ea-35dd-4707-b63f-78b5d638c1b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:44.875685 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d681d4ea-35dd-4707-b63f-78b5d638c1b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:44.877219 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d681d4ea-35dd-4707-b63f-78b5d638c1b2 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:44.901803 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-d681d4ea-35dd-4707-b63f-78b5d638c1b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:30:44.901803 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:30:44.901803 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:30:44.901803 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:30:44.901803 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:30:44.901803 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:30:44.901803 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:30:44.901803 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:30:44.901803 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:30:44.901803 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:30:44.901803 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:30:44.901803 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:30:44.901803 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:30:44.901803 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:30:44.901803 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:30:44.903891 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:30:44.903891 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:30:44.903891 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:30:44.903891 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:30:44.903891 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:30:44.903891 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:30:44.903891 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:30:44.903891 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:30:44.903891 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:30:44.903891 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:30:44.903891 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:30:44.903891 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:30:44.903891 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:30:44.903891 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:30:44.903891 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:30:44.903891 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:30:44.903891 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:30:44.904886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:30:44.904886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:30:44.904886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:30:44.904886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:30:44.904886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 616/1210] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:30:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:30:44.910335 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53fa3b0-3f08-4e5c-882c-b0d844c721b3 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:45.054060 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:45.060069 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53fa3b0-3f08-4e5c-882c-b0d844c721b3 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:45.115089 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1af6aa-5e68-45a8-8e3d-7daf7029c77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:45.116106 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1af6aa-5e68-45a8-8e3d-7daf7029c77f None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:45.116538 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1af6aa-5e68-45a8-8e3d-7daf7029c77f None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:45.116971 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1af6aa-5e68-45a8-8e3d-7daf7029c77f None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:45.119613 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1af6aa-5e68-45a8-8e3d-7daf7029c77f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:45.119613 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1af6aa-5e68-45a8-8e3d-7daf7029c77f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:45.120908 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1af6aa-5e68-45a8-8e3d-7daf7029c77f None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:45.144045 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 595/1211] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:30:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:45.145941 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d681d4ea-35dd-4707-b63f-78b5d638c1b2 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:45.331774 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d681d4ea-35dd-4707-b63f-78b5d638c1b2 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:45.334531 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:45.387659 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b357d-9932-45e9-91de-60c5551c6d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:45.388501 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b357d-9932-45e9-91de-60c5551c6d6e None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:45.388770 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b357d-9932-45e9-91de-60c5551c6d6e None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:45.389072 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b357d-9932-45e9-91de-60c5551c6d6e None admin] PATH_INFO: `/v3/users/placement` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:45.397295 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809b357d-9932-45e9-91de-60c5551c6d6e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:45.398807 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809b357d-9932-45e9-91de-60c5551c6d6e None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:45.415239 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-809b357d-9932-45e9-91de-60c5551c6d6e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 08:30:45.415239 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:30:45.415239 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:30:45.415239 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:30:45.415239 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:30:45.415239 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:30:45.415239 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:30:45.415239 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:30:45.415239 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:30:45.415239 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:30:45.415239 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:30:45.415239 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:30:45.415239 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:30:45.415239 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:30:45.415239 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:30:45.415239 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:30:45.417634 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:30:45.417634 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:30:45.417634 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:30:45.417634 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:30:45.417634 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:30:45.417634 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:30:45.417634 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:30:45.417634 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:30:45.417634 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:30:45.417634 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:30:45.417634 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:30:45.417634 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:30:45.417634 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:30:45.417634 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:30:45.417634 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:30:45.417634 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:30:45.417634 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:30:45.417634 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:30:45.419480 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:30:45.419480 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:30:45.419480 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:30:45.419480 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:30:45.419480 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:30:45.419480 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:30:45.419480 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:30:45.419480 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:30:45.419480 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 08:30:45.419480 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:30:45.419480 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 617/1212] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:30:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:30:45.423213 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1af6aa-5e68-45a8-8e3d-7daf7029c77f None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:45.565161 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1af6aa-5e68-45a8-8e3d-7daf7029c77f None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:45.611891 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c206cd0d-6e4b-481a-b276-81ec1c2fe6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:45.612860 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206cd0d-6e4b-481a-b276-81ec1c2fe6bc None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:45.613264 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206cd0d-6e4b-481a-b276-81ec1c2fe6bc None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:45.613814 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206cd0d-6e4b-481a-b276-81ec1c2fe6bc None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:45.614975 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c206cd0d-6e4b-481a-b276-81ec1c2fe6bc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:45.615384 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c206cd0d-6e4b-481a-b276-81ec1c2fe6bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:45.618676 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c206cd0d-6e4b-481a-b276-81ec1c2fe6bc None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:45.649818 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 596/1213] 10.222.0.13 () {56 vars in 1143 bytes} [Fri Jun 24 08:30:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:45.652200 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b357d-9932-45e9-91de-60c5551c6d6e None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:45.776412 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b357d-9932-45e9-91de-60c5551c6d6e None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:45.811816 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f83f5-107f-4b47-ad27-216785291a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:45.812486 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f83f5-107f-4b47-ad27-216785291a58 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:45.812801 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f83f5-107f-4b47-ad27-216785291a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:45.813024 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f83f5-107f-4b47-ad27-216785291a58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:45.851340 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:45.899704 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3f83f5-107f-4b47-ad27-216785291a58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:45.901272 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3f83f5-107f-4b47-ad27-216785291a58 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:46.028926 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 618/1214] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:30:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 08:30:46.035961 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c206cd0d-6e4b-481a-b276-81ec1c2fe6bc None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:46.070049 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:46.177983 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c206cd0d-6e4b-481a-b276-81ec1c2fe6bc None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:46.215344 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a451607-9cef-43b9-87e0-7c2e29e7cfe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:46.216670 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a451607-9cef-43b9-87e0-7c2e29e7cfe3 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:46.217138 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a451607-9cef-43b9-87e0-7c2e29e7cfe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:46.217626 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a451607-9cef-43b9-87e0-7c2e29e7cfe3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:46.232369 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a451607-9cef-43b9-87e0-7c2e29e7cfe3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:46.235933 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a451607-9cef-43b9-87e0-7c2e29e7cfe3 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:46.260813 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-0a451607-9cef-43b9-87e0-7c2e29e7cfe3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:30:46.260813 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:30:46.260813 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:30:46.260813 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:30:46.260813 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:30:46.260813 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:30:46.260813 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:30:46.260813 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:30:46.260813 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:30:46.260813 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:30:46.260813 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:30:46.260813 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:30:46.260813 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:30:46.260813 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:30:46.260813 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:30:46.261879 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:30:46.261879 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:30:46.261879 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:30:46.261879 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:30:46.261879 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:30:46.261879 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:30:46.261879 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:30:46.261879 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:30:46.261879 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:30:46.261879 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:30:46.261879 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:30:46.261879 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:30:46.261879 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:30:46.261879 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:30:46.261879 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:30:46.261879 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:30:46.261879 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:30:46.262751 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:30:46.262751 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:30:46.262751 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:30:46.262751 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:30:46.262751 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 597/1215] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:30:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:30:46.268378 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f83f5-107f-4b47-ad27-216785291a58 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:46.370772 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:46.420246 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f83f5-107f-4b47-ad27-216785291a58 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:46.465255 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b5c8ae-8cd7-4987-a6f8-870a02cacfb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:46.465883 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b5c8ae-8cd7-4987-a6f8-870a02cacfb7 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:46.466169 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b5c8ae-8cd7-4987-a6f8-870a02cacfb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:46.466446 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b5c8ae-8cd7-4987-a6f8-870a02cacfb7 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:46.467167 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b5c8ae-8cd7-4987-a6f8-870a02cacfb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:46.467474 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b5c8ae-8cd7-4987-a6f8-870a02cacfb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:46.468846 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b5c8ae-8cd7-4987-a6f8-870a02cacfb7 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:46.485408 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 619/1216] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:30:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:46.492542 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a451607-9cef-43b9-87e0-7c2e29e7cfe3 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:46.586256 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:46.622071 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a451607-9cef-43b9-87e0-7c2e29e7cfe3 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:46.667391 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb914de8-e454-42fd-ac75-f32c0e6d6d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:46.668301 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb914de8-e454-42fd-ac75-f32c0e6d6d3e None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:46.668633 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb914de8-e454-42fd-ac75-f32c0e6d6d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:46.668983 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb914de8-e454-42fd-ac75-f32c0e6d6d3e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:46.677266 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb914de8-e454-42fd-ac75-f32c0e6d6d3e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:46.679002 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb914de8-e454-42fd-ac75-f32c0e6d6d3e None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:46.699577 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-fb914de8-e454-42fd-ac75-f32c0e6d6d3e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:30:46.699577 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:30:46.699577 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:30:46.699577 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:30:46.699577 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:30:46.699577 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:30:46.699577 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:30:46.699577 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:30:46.699577 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:30:46.699577 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:30:46.699577 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:30:46.699577 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:30:46.699577 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:30:46.699577 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:30:46.699577 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:30:46.700985 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:30:46.700985 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:30:46.700985 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:30:46.700985 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:30:46.700985 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:30:46.700985 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:30:46.700985 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:30:46.700985 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:30:46.700985 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:30:46.700985 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:30:46.700985 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:30:46.700985 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:30:46.700985 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:30:46.700985 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:30:46.700985 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:30:46.700985 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:30:46.700985 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:30:46.707427 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:30:46.707427 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:30:46.707427 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:30:46.707427 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:30:46.707427 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:30:46.707427 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:30:46.707427 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:30:46.707427 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:30:46.707427 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 598/1217] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:30:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:30:46.709648 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b5c8ae-8cd7-4987-a6f8-870a02cacfb7 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:46.842357 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b5c8ae-8cd7-4987-a6f8-870a02cacfb7 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:46.871622 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:46.888819 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02af190b-822e-464b-9d1b-a08b441e67c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:46.889629 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af190b-822e-464b-9d1b-a08b441e67c3 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:46.889996 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af190b-822e-464b-9d1b-a08b441e67c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:46.890489 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af190b-822e-464b-9d1b-a08b441e67c3 None admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:46.891467 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02af190b-822e-464b-9d1b-a08b441e67c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:46.891852 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02af190b-822e-464b-9d1b-a08b441e67c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:46.893588 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02af190b-822e-464b-9d1b-a08b441e67c3 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:46.911809 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 620/1218] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Jun 24 08:30:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:46.918916 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb914de8-e454-42fd-ac75-f32c0e6d6d3e None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:47.056668 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb914de8-e454-42fd-ac75-f32c0e6d6d3e None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:47.090933 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:47.098608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578be0d5-313e-4d2e-a275-b5b3e8eb8ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:47.099436 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578be0d5-313e-4d2e-a275-b5b3e8eb8ed4 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:47.099785 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578be0d5-313e-4d2e-a275-b5b3e8eb8ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:47.100119 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578be0d5-313e-4d2e-a275-b5b3e8eb8ed4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:47.101200 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578be0d5-313e-4d2e-a275-b5b3e8eb8ed4 None admin] RBAC: Adding query filter params (role.id=6ca1ae4292d9414ca6526c6e8608de9e, scope.project.id=180b0af5863349e99ae412b07c1e9465, user.id=39cc268bad604dd08594e3b39c7ecf5a) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:47.101578 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578be0d5-313e-4d2e-a275-b5b3e8eb8ed4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:47.103950 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578be0d5-313e-4d2e-a275-b5b3e8eb8ed4 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:47.110730 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 599/1219] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Jun 24 08:30:46 2022] GET /identity/v3/role_assignments?user.id=39cc268bad604dd08594e3b39c7ecf5a&scope.project.id=180b0af5863349e99ae412b07c1e9465&role.id=6ca1ae4292d9414ca6526c6e8608de9e => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:30:49.280271 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662755da-54ad-4a48-aafe-4590903cb078 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:49.280882 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662755da-54ad-4a48-aafe-4590903cb078 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:49.280882 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662755da-54ad-4a48-aafe-4590903cb078 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:49.282217 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 621/1220] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:30:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:30:49.287777 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae35f6-2d49-44c8-8864-cb88cc4ebec4 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:49.288455 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae35f6-2d49-44c8-8864-cb88cc4ebec4 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:49.288455 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae35f6-2d49-44c8-8864-cb88cc4ebec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:49.309818 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-0dae35f6-2d49-44c8-8864-cb88cc4ebec4 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:30:49.310989 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-0dae35f6-2d49-44c8-8864-cb88cc4ebec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:30:49.386946 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 600/1221] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:30:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:30:49.412294 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662755da-54ad-4a48-aafe-4590903cb078 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:49.535043 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662755da-54ad-4a48-aafe-4590903cb078 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:49.576189 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad3a76b-91b0-4259-8387-f73d4a5da353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:49.577202 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3a76b-91b0-4259-8387-f73d4a5da353 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:49.577489 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3a76b-91b0-4259-8387-f73d4a5da353 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:49.577874 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3a76b-91b0-4259-8387-f73d4a5da353 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:49.589315 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad3a76b-91b0-4259-8387-f73d4a5da353 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:30:49.597073 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-cad3a76b-91b0-4259-8387-f73d4a5da353 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:30:49.597073 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:30:49.597073 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:30:49.597073 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:30:49.597073 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:30:49.597073 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:30:49.597073 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:30:49.597073 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:30:49.597073 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:30:49.597073 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:30:49.597073 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:30:49.597073 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:30:49.597073 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:30:49.597073 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:30:49.597073 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:30:49.597073 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:30:49.598162 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:30:49.598162 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:30:49.598162 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:30:49.598162 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:30:49.598162 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:30:49.598162 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:30:49.598162 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:30:49.598162 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:30:49.598162 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:30:49.598162 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:30:49.598162 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:30:49.598162 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:30:49.598162 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:30:49.598162 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:30:49.598162 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:30:49.598162 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:30:49.599030 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:30:49.599030 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:30:49.599030 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:30:49.599030 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:30:49.599030 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:30:49.599030 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:30:49.599030 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:30:49.599030 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:30:49.599030 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:30:49.599030 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:30:49.599030 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:30:49.599030 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:30:49.599030 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 622/1222] 10.222.0.13 () {56 vars in 1102 bytes} [Fri Jun 24 08:30:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:30:49.606058 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dae35f6-2d49-44c8-8864-cb88cc4ebec4 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:49.723410 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dae35f6-2d49-44c8-8864-cb88cc4ebec4 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:49.749548 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:49.762248 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945cae06-d048-46e4-9353-ae28dae22bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:49.762938 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945cae06-d048-46e4-9353-ae28dae22bc9 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:49.763369 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945cae06-d048-46e4-9353-ae28dae22bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:49.763778 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945cae06-d048-46e4-9353-ae28dae22bc9 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:49.764874 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945cae06-d048-46e4-9353-ae28dae22bc9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:49.765308 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945cae06-d048-46e4-9353-ae28dae22bc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:49.766894 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945cae06-d048-46e4-9353-ae28dae22bc9 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:49.778509 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 601/1223] 10.222.0.13 () {56 vars in 1103 bytes} [Fri Jun 24 08:30:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:30:49.784286 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad3a76b-91b0-4259-8387-f73d4a5da353 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:49.840625 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:49.919545 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad3a76b-91b0-4259-8387-f73d4a5da353 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:49.963260 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bd4305-0b27-42eb-b376-291bb8bf7a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:49.964428 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd4305-0b27-42eb-b376-291bb8bf7a21 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:49.964715 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd4305-0b27-42eb-b376-291bb8bf7a21 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:49.964996 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd4305-0b27-42eb-b376-291bb8bf7a21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:49.977963 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bd4305-0b27-42eb-b376-291bb8bf7a21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:49.980039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bd4305-0b27-42eb-b376-291bb8bf7a21 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:49.998833 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-65bd4305-0b27-42eb-b376-291bb8bf7a21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:30:49.998833 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:30:49.998833 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:30:49.998833 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:30:49.998833 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:30:49.998833 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:30:49.998833 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:30:49.998833 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:30:49.998833 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:30:49.998833 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:30:49.998833 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:30:49.998833 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:30:49.998833 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:30:49.998833 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:30:49.998833 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:30:49.999924 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:30:49.999924 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:30:49.999924 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:30:49.999924 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:30:49.999924 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:30:49.999924 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:30:49.999924 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:30:49.999924 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:30:49.999924 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:30:49.999924 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:30:49.999924 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:30:49.999924 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:30:49.999924 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:30:49.999924 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:30:49.999924 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:30:49.999924 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:30:49.999924 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:30:50.000980 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:30:50.000980 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:30:50.000980 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:30:50.000980 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:30:50.000980 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 623/1224] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:30:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:30:50.005905 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945cae06-d048-46e4-9353-ae28dae22bc9 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:50.123500 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945cae06-d048-46e4-9353-ae28dae22bc9 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:50.159808 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c273f-c1a9-4e65-9fa2-f77bb0adf1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:50.160675 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c273f-c1a9-4e65-9fa2-f77bb0adf1e6 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:50.161023 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c273f-c1a9-4e65-9fa2-f77bb0adf1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:50.161447 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c273f-c1a9-4e65-9fa2-f77bb0adf1e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:50.162298 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246c273f-c1a9-4e65-9fa2-f77bb0adf1e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:50.162609 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246c273f-c1a9-4e65-9fa2-f77bb0adf1e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:50.164506 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246c273f-c1a9-4e65-9fa2-f77bb0adf1e6 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:50.178614 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 602/1225] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:30:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:50.184876 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bd4305-0b27-42eb-b376-291bb8bf7a21 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:50.296788 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:50.304065 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bd4305-0b27-42eb-b376-291bb8bf7a21 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:50.343335 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc3a14e-e833-45c9-ae9a-eacd2814db2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:50.344153 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc3a14e-e833-45c9-ae9a-eacd2814db2a None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:50.344439 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc3a14e-e833-45c9-ae9a-eacd2814db2a None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:50.344714 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc3a14e-e833-45c9-ae9a-eacd2814db2a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:50.354190 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc3a14e-e833-45c9-ae9a-eacd2814db2a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:50.355621 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc3a14e-e833-45c9-ae9a-eacd2814db2a None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:50.372329 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-9dc3a14e-e833-45c9-ae9a-eacd2814db2a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 08:30:50.372329 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:30:50.372329 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:30:50.372329 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:30:50.372329 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:30:50.372329 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:30:50.372329 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:30:50.372329 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:30:50.372329 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:30:50.372329 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:30:50.372329 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:30:50.372329 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:30:50.372329 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:30:50.372329 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:30:50.372329 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:30:50.372329 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:30:50.373644 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:30:50.373644 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:30:50.373644 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:30:50.373644 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:30:50.373644 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:30:50.373644 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:30:50.373644 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:30:50.373644 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:30:50.373644 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:30:50.373644 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:30:50.373644 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:30:50.373644 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:30:50.373644 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:30:50.373644 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:30:50.373644 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:30:50.373644 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:30:50.373644 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:30:50.373644 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:30:50.374825 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:30:50.374825 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:30:50.374825 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:30:50.374825 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:30:50.374825 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:30:50.374825 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:30:50.374825 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:30:50.374825 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:30:50.374825 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 08:30:50.374825 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:30:50.374825 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 624/1226] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:30:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:30:50.379298 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c273f-c1a9-4e65-9fa2-f77bb0adf1e6 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:50.435396 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:50.506050 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c273f-c1a9-4e65-9fa2-f77bb0adf1e6 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:50.544037 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac205d1e-f238-4677-b335-c7f45671577a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:50.544694 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac205d1e-f238-4677-b335-c7f45671577a None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:50.544975 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac205d1e-f238-4677-b335-c7f45671577a None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:50.545252 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac205d1e-f238-4677-b335-c7f45671577a None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:50.546451 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac205d1e-f238-4677-b335-c7f45671577a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:50.546775 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac205d1e-f238-4677-b335-c7f45671577a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:50.548431 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac205d1e-f238-4677-b335-c7f45671577a None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:50.567881 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 603/1227] 10.222.0.13 () {56 vars in 1143 bytes} [Fri Jun 24 08:30:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:50.576212 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc3a14e-e833-45c9-ae9a-eacd2814db2a None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:50.692489 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc3a14e-e833-45c9-ae9a-eacd2814db2a None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:50.734902 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09681aed-070a-4268-ab14-1fcaa563e810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:50.735665 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09681aed-070a-4268-ab14-1fcaa563e810 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:50.735954 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09681aed-070a-4268-ab14-1fcaa563e810 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:50.736229 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09681aed-070a-4268-ab14-1fcaa563e810 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:50.748669 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09681aed-070a-4268-ab14-1fcaa563e810 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:50.750180 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09681aed-070a-4268-ab14-1fcaa563e810 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:50.771067 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-09681aed-070a-4268-ab14-1fcaa563e810 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:30:50.771067 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:30:50.771067 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:30:50.771067 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:30:50.771067 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:30:50.771067 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:30:50.771067 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:30:50.771067 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:30:50.771067 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:30:50.771067 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:30:50.771067 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:30:50.771067 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:30:50.771067 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:30:50.771067 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:30:50.771067 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:30:50.772150 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:30:50.772150 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:30:50.772150 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:30:50.772150 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:30:50.772150 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:30:50.772150 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:30:50.772150 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:30:50.772150 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:30:50.772150 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:30:50.772150 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:30:50.772150 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:30:50.772150 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:30:50.772150 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:30:50.772150 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:30:50.772150 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:30:50.772150 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:30:50.772150 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:30:50.773301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:30:50.773301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:30:50.773301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:30:50.773301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:30:50.773301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 625/1228] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:30:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:30:50.779116 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac205d1e-f238-4677-b335-c7f45671577a None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:50.898070 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac205d1e-f238-4677-b335-c7f45671577a None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:50.901245 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:50.941069 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd554c96-f779-456f-ae6e-e227b2c7e674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:50.942314 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd554c96-f779-456f-ae6e-e227b2c7e674 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:50.942771 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd554c96-f779-456f-ae6e-e227b2c7e674 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:50.943069 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd554c96-f779-456f-ae6e-e227b2c7e674 None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:50.943895 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd554c96-f779-456f-ae6e-e227b2c7e674 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:50.944200 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd554c96-f779-456f-ae6e-e227b2c7e674 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:50.945875 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd554c96-f779-456f-ae6e-e227b2c7e674 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:50.961245 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 604/1229] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:30:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:30:50.967931 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09681aed-070a-4268-ab14-1fcaa563e810 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:50.978793 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:51.080516 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09681aed-070a-4268-ab14-1fcaa563e810 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:51.126294 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8223b2-a657-4100-add2-09ea02fc8da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:51.127102 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8223b2-a657-4100-add2-09ea02fc8da6 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:51.127384 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8223b2-a657-4100-add2-09ea02fc8da6 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:51.127656 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8223b2-a657-4100-add2-09ea02fc8da6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:51.136303 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8223b2-a657-4100-add2-09ea02fc8da6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:51.137876 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8223b2-a657-4100-add2-09ea02fc8da6 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:51.156529 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-dd8223b2-a657-4100-add2-09ea02fc8da6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:30:51.156529 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:30:51.156529 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:30:51.156529 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:30:51.156529 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:30:51.156529 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:30:51.156529 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:30:51.156529 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:30:51.156529 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:30:51.156529 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:30:51.156529 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:30:51.156529 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:30:51.156529 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:30:51.156529 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:30:51.156529 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:30:51.157668 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:30:51.157668 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:30:51.157668 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:30:51.157668 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:30:51.157668 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:30:51.157668 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:30:51.157668 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:30:51.157668 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:30:51.157668 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:30:51.157668 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:30:51.157668 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:30:51.157668 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:30:51.157668 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:30:51.157668 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:30:51.157668 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:30:51.157668 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:30:51.157668 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:30:51.158765 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:30:51.158765 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:30:51.158765 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:30:51.158765 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:30:51.158765 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:30:51.158765 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:30:51.158765 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:30:51.158765 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:30:51.159188 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 626/1230] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:30:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:30:51.164553 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd554c96-f779-456f-ae6e-e227b2c7e674 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:51.292257 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd554c96-f779-456f-ae6e-e227b2c7e674 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:51.328886 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabdf99a-0020-4906-85ff-fcb6b7482827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:51.329815 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabdf99a-0020-4906-85ff-fcb6b7482827 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:51.330228 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabdf99a-0020-4906-85ff-fcb6b7482827 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:51.330535 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabdf99a-0020-4906-85ff-fcb6b7482827 None admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:51.331452 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabdf99a-0020-4906-85ff-fcb6b7482827 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:51.331793 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabdf99a-0020-4906-85ff-fcb6b7482827 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:51.333390 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabdf99a-0020-4906-85ff-fcb6b7482827 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:51.347749 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 605/1231] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Jun 24 08:30:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:51.355196 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8223b2-a657-4100-add2-09ea02fc8da6 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:51.432976 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:51.480119 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8223b2-a657-4100-add2-09ea02fc8da6 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:51.529570 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952ccb54-5ed0-43e0-a95e-76d920930ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:51.530763 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ccb54-5ed0-43e0-a95e-76d920930ede None admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:51.531807 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ccb54-5ed0-43e0-a95e-76d920930ede None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:51.532191 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ccb54-5ed0-43e0-a95e-76d920930ede None admin] PATH_INFO: `/v3/projects/180b0af5863349e99ae412b07c1e9465/users/39cc268bad604dd08594e3b39c7ecf5a/roles/6ca1ae4292d9414ca6526c6e8608de9e` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:51.551712 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952ccb54-5ed0-43e0-a95e-76d920930ede None admin] RBAC: Authorizing `identity:create_grant(project_id=180b0af5863349e99ae412b07c1e9465, user_id=39cc268bad604dd08594e3b39c7ecf5a, role_id=6ca1ae4292d9414ca6526c6e8608de9e)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:51.553200 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952ccb54-5ed0-43e0-a95e-76d920930ede None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:51.573197 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 627/1232] 10.222.0.13 () {58 vars in 1439 bytes} [Fri Jun 24 08:30:51 2022] PUT /identity/v3/projects/180b0af5863349e99ae412b07c1e9465/users/39cc268bad604dd08594e3b39c7ecf5a/roles/6ca1ae4292d9414ca6526c6e8608de9e => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:30:53.842479 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97c279-037c-46ec-8649-7388a0e9c167 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:53.843000 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97c279-037c-46ec-8649-7388a0e9c167 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:53.843091 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97c279-037c-46ec-8649-7388a0e9c167 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:53.848516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 606/1233] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:30:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:30:53.850281 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a73ddf-c9a2-4589-864a-56d13f91150c None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:53.850713 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a73ddf-c9a2-4589-864a-56d13f91150c None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:53.850888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a73ddf-c9a2-4589-864a-56d13f91150c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:53.871690 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-69a73ddf-c9a2-4589-864a-56d13f91150c None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:30:53.873348 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-69a73ddf-c9a2-4589-864a-56d13f91150c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:30:53.962085 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 628/1234] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:30:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:30:53.987873 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb97c279-037c-46ec-8649-7388a0e9c167 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:54.020045 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:54.134049 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb97c279-037c-46ec-8649-7388a0e9c167 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:54.183145 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f491f936-e182-47b8-bdc4-6a3d5f56f393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:54.184261 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f491f936-e182-47b8-bdc4-6a3d5f56f393 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:54.184683 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f491f936-e182-47b8-bdc4-6a3d5f56f393 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:54.185119 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f491f936-e182-47b8-bdc4-6a3d5f56f393 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:54.198448 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f491f936-e182-47b8-bdc4-6a3d5f56f393 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:30:54.207362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-f491f936-e182-47b8-bdc4-6a3d5f56f393 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:30:54.207362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:30:54.207362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:30:54.207362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:30:54.207362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:30:54.207362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:30:54.207362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:30:54.207362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:30:54.207362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:30:54.207362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:30:54.207362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:30:54.207362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:30:54.207362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:30:54.207362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:30:54.207362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:30:54.207362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:30:54.208416 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:30:54.208416 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:30:54.208416 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:30:54.208416 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:30:54.208416 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:30:54.208416 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:30:54.208416 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:30:54.208416 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:30:54.208416 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:30:54.208416 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:30:54.208416 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:30:54.208416 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:30:54.208416 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:30:54.208416 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:30:54.208416 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:30:54.208416 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:30:54.209537 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:30:54.209537 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:30:54.209537 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:30:54.209537 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:30:54.209537 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:30:54.209537 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:30:54.209537 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:30:54.209537 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:30:54.209537 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:30:54.209537 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:30:54.209537 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:30:54.209537 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:30:54.209537 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 607/1235] 10.222.0.13 () {56 vars in 1102 bytes} [Fri Jun 24 08:30:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:30:54.215683 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a73ddf-c9a2-4589-864a-56d13f91150c None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:54.276880 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:54.345342 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a73ddf-c9a2-4589-864a-56d13f91150c None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:54.394734 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9090f86c-9d92-4d64-8f39-3984c0ac71e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:54.395421 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9090f86c-9d92-4d64-8f39-3984c0ac71e6 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:54.395971 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9090f86c-9d92-4d64-8f39-3984c0ac71e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:54.396418 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9090f86c-9d92-4d64-8f39-3984c0ac71e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:54.397170 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9090f86c-9d92-4d64-8f39-3984c0ac71e6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:54.397486 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9090f86c-9d92-4d64-8f39-3984c0ac71e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:54.399031 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9090f86c-9d92-4d64-8f39-3984c0ac71e6 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:54.410440 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 629/1236] 10.222.0.13 () {56 vars in 1103 bytes} [Fri Jun 24 08:30:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:30:54.416406 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f491f936-e182-47b8-bdc4-6a3d5f56f393 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:54.532036 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:54.552672 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f491f936-e182-47b8-bdc4-6a3d5f56f393 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:54.594298 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3aa9e7-1d2c-4d9c-a16c-e652cd7f77c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:54.595095 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3aa9e7-1d2c-4d9c-a16c-e652cd7f77c6 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:54.595390 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3aa9e7-1d2c-4d9c-a16c-e652cd7f77c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:54.595691 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3aa9e7-1d2c-4d9c-a16c-e652cd7f77c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:54.609193 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3aa9e7-1d2c-4d9c-a16c-e652cd7f77c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:54.610728 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3aa9e7-1d2c-4d9c-a16c-e652cd7f77c6 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:54.630928 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-6b3aa9e7-1d2c-4d9c-a16c-e652cd7f77c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:30:54.630928 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:30:54.630928 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:30:54.630928 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:30:54.630928 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:30:54.630928 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:30:54.630928 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:30:54.630928 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:30:54.630928 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:30:54.630928 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:30:54.630928 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:30:54.630928 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:30:54.630928 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:30:54.630928 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:30:54.630928 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:30:54.633372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:30:54.633372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:30:54.633372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:30:54.633372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:30:54.633372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:30:54.633372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:30:54.633372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:30:54.633372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:30:54.633372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:30:54.633372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:30:54.633372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:30:54.633372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:30:54.633372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:30:54.633372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:30:54.633372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:30:54.633372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:30:54.633372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:30:54.635722 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:30:54.635722 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:30:54.635722 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:30:54.635722 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:30:54.635722 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 608/1237] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:30:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:30:54.639399 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9090f86c-9d92-4d64-8f39-3984c0ac71e6 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:54.763401 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9090f86c-9d92-4d64-8f39-3984c0ac71e6 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:54.774134 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:54.803288 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d928d69-dc17-4831-8c21-9a7b761fec1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:54.804066 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d928d69-dc17-4831-8c21-9a7b761fec1c None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:54.804384 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d928d69-dc17-4831-8c21-9a7b761fec1c None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:54.804730 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d928d69-dc17-4831-8c21-9a7b761fec1c None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:54.805540 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d928d69-dc17-4831-8c21-9a7b761fec1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:54.805875 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d928d69-dc17-4831-8c21-9a7b761fec1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:54.807997 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d928d69-dc17-4831-8c21-9a7b761fec1c None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:54.821560 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 630/1238] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:30:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:54.827304 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3aa9e7-1d2c-4d9c-a16c-e652cd7f77c6 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:54.950440 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3aa9e7-1d2c-4d9c-a16c-e652cd7f77c6 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:54.993003 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19b43a6-383f-422d-966f-ed881b2de8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:54.994456 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19b43a6-383f-422d-966f-ed881b2de8c8 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:54.994980 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19b43a6-383f-422d-966f-ed881b2de8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:54.995233 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19b43a6-383f-422d-966f-ed881b2de8c8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:54.998744 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:55.007365 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19b43a6-383f-422d-966f-ed881b2de8c8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:55.009322 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19b43a6-383f-422d-966f-ed881b2de8c8 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:55.028727 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-f19b43a6-383f-422d-966f-ed881b2de8c8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 08:30:55.028727 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:30:55.028727 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:30:55.028727 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:30:55.028727 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:30:55.028727 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:30:55.028727 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:30:55.028727 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:30:55.028727 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:30:55.028727 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:30:55.028727 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:30:55.028727 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:30:55.028727 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:30:55.028727 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:30:55.028727 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:30:55.028727 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:30:55.029976 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:30:55.029976 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:30:55.029976 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:30:55.029976 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:30:55.029976 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:30:55.029976 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:30:55.029976 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:30:55.029976 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:30:55.029976 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:30:55.029976 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:30:55.029976 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:30:55.029976 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:30:55.029976 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:30:55.029976 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:30:55.029976 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:30:55.029976 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:30:55.029976 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:30:55.029976 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:30:55.030969 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:30:55.030969 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:30:55.030969 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:30:55.030969 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:30:55.030969 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:30:55.030969 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:30:55.030969 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:30:55.030969 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:30:55.030969 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 08:30:55.030969 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:30:55.030969 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 609/1239] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:30:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:30:55.036570 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d928d69-dc17-4831-8c21-9a7b761fec1c None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:55.165327 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d928d69-dc17-4831-8c21-9a7b761fec1c None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:55.209226 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19395e20-272e-4449-ae5d-ed4114ef9f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:55.210011 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19395e20-272e-4449-ae5d-ed4114ef9f07 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:55.210364 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19395e20-272e-4449-ae5d-ed4114ef9f07 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:55.210700 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19395e20-272e-4449-ae5d-ed4114ef9f07 None admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:55.211579 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19395e20-272e-4449-ae5d-ed4114ef9f07 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:55.211962 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19395e20-272e-4449-ae5d-ed4114ef9f07 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:55.213568 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19395e20-272e-4449-ae5d-ed4114ef9f07 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:55.233638 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 631/1240] 10.222.0.13 () {56 vars in 1143 bytes} [Fri Jun 24 08:30:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:55.239244 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19b43a6-383f-422d-966f-ed881b2de8c8 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:55.368344 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19b43a6-383f-422d-966f-ed881b2de8c8 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:55.416680 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d83a783-7d25-4125-b141-745b7dfc73d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:55.417393 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d83a783-7d25-4125-b141-745b7dfc73d2 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:55.417829 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d83a783-7d25-4125-b141-745b7dfc73d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:55.418047 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d83a783-7d25-4125-b141-745b7dfc73d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:55.502928 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:55.511337 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d83a783-7d25-4125-b141-745b7dfc73d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:55.512807 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d83a783-7d25-4125-b141-745b7dfc73d2 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:55.641360 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 610/1241] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:30:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 08:30:55.647497 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19395e20-272e-4449-ae5d-ed4114ef9f07 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:55.683717 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:55.783814 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19395e20-272e-4449-ae5d-ed4114ef9f07 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:55.825363 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc55e59-fdf2-49f4-8919-053fe8932476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:55.826269 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc55e59-fdf2-49f4-8919-053fe8932476 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:55.826549 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc55e59-fdf2-49f4-8919-053fe8932476 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:55.826824 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc55e59-fdf2-49f4-8919-053fe8932476 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:55.838622 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc55e59-fdf2-49f4-8919-053fe8932476 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:55.840194 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc55e59-fdf2-49f4-8919-053fe8932476 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:55.861201 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-ccc55e59-fdf2-49f4-8919-053fe8932476 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:30:55.861201 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:30:55.861201 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:30:55.861201 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:30:55.861201 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:30:55.861201 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:30:55.861201 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:30:55.861201 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:30:55.861201 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:30:55.861201 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:30:55.861201 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:30:55.861201 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:30:55.861201 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:30:55.861201 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:30:55.861201 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:30:55.862501 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:30:55.862501 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:30:55.862501 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:30:55.862501 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:30:55.862501 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:30:55.862501 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:30:55.862501 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:30:55.862501 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:30:55.862501 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:30:55.862501 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:30:55.862501 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:30:55.862501 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:30:55.862501 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:30:55.862501 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:30:55.862501 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:30:55.862501 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:30:55.862501 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:30:55.863615 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:30:55.863615 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:30:55.863615 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:30:55.863615 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:30:55.863615 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 632/1242] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:30:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:30:55.868457 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d83a783-7d25-4125-b141-745b7dfc73d2 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:55.995275 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:56.003374 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d83a783-7d25-4125-b141-745b7dfc73d2 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:56.044687 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d741bb6-f42f-4da8-8dab-498fceae0f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:56.045474 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d741bb6-f42f-4da8-8dab-498fceae0f80 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:56.045814 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d741bb6-f42f-4da8-8dab-498fceae0f80 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:56.047705 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d741bb6-f42f-4da8-8dab-498fceae0f80 None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:56.047705 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d741bb6-f42f-4da8-8dab-498fceae0f80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:56.047705 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d741bb6-f42f-4da8-8dab-498fceae0f80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:56.048550 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d741bb6-f42f-4da8-8dab-498fceae0f80 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:56.065471 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 611/1243] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:30:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:56.071163 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc55e59-fdf2-49f4-8919-053fe8932476 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:56.172225 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:56.197547 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc55e59-fdf2-49f4-8919-053fe8932476 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:56.236665 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfcb3b1-1a65-4ae1-88ee-d918fe44160c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:56.237528 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfcb3b1-1a65-4ae1-88ee-d918fe44160c None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:56.237850 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfcb3b1-1a65-4ae1-88ee-d918fe44160c None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:56.238146 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfcb3b1-1a65-4ae1-88ee-d918fe44160c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:56.246226 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfcb3b1-1a65-4ae1-88ee-d918fe44160c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:56.247635 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfcb3b1-1a65-4ae1-88ee-d918fe44160c None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:56.263827 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-0bfcb3b1-1a65-4ae1-88ee-d918fe44160c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:30:56.263827 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:30:56.263827 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:30:56.263827 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:30:56.263827 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:30:56.263827 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:30:56.263827 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:30:56.263827 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:30:56.263827 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:30:56.263827 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:30:56.263827 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:30:56.263827 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:30:56.263827 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:30:56.263827 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:30:56.263827 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:30:56.264976 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:30:56.264976 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:30:56.264976 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:30:56.264976 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:30:56.264976 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:30:56.264976 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:30:56.264976 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:30:56.264976 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:30:56.264976 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:30:56.264976 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:30:56.264976 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:30:56.264976 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:30:56.264976 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:30:56.264976 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:30:56.264976 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:30:56.264976 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:30:56.264976 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:30:56.265937 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:30:56.265937 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:30:56.265937 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:30:56.265937 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:30:56.265937 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:30:56.265937 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:30:56.265937 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:30:56.265937 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:30:56.265937 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 633/1244] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:30:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:30:56.271201 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d741bb6-f42f-4da8-8dab-498fceae0f80 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:56.399806 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d741bb6-f42f-4da8-8dab-498fceae0f80 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:56.442915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa4d15-c78f-4e2a-91ed-5dddbd67bebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:56.443666 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa4d15-c78f-4e2a-91ed-5dddbd67bebf None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:56.443950 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa4d15-c78f-4e2a-91ed-5dddbd67bebf None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:56.444254 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa4d15-c78f-4e2a-91ed-5dddbd67bebf None admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:56.445041 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daa4d15-c78f-4e2a-91ed-5dddbd67bebf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:56.445349 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daa4d15-c78f-4e2a-91ed-5dddbd67bebf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:56.446845 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daa4d15-c78f-4e2a-91ed-5dddbd67bebf None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:56.466299 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 612/1245] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Jun 24 08:30:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:56.473761 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfcb3b1-1a65-4ae1-88ee-d918fe44160c None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:56.606867 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfcb3b1-1a65-4ae1-88ee-d918fe44160c None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:56.648162 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4e63a7-7813-456b-8ba2-58959ddb0fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:56.648856 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4e63a7-7813-456b-8ba2-58959ddb0fed None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:56.649149 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4e63a7-7813-456b-8ba2-58959ddb0fed None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:56.649473 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4e63a7-7813-456b-8ba2-58959ddb0fed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:56.650378 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4e63a7-7813-456b-8ba2-58959ddb0fed None admin] RBAC: Adding query filter params (role.id=6ca1ae4292d9414ca6526c6e8608de9e, scope.project.id=180b0af5863349e99ae412b07c1e9465, user.id=39cc268bad604dd08594e3b39c7ecf5a) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:56.650742 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4e63a7-7813-456b-8ba2-58959ddb0fed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:56.652147 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4e63a7-7813-456b-8ba2-58959ddb0fed None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:56.658845 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 634/1246] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Jun 24 08:30:56 2022] GET /identity/v3/role_assignments?user.id=39cc268bad604dd08594e3b39c7ecf5a&scope.project.id=180b0af5863349e99ae412b07c1e9465&role.id=6ca1ae4292d9414ca6526c6e8608de9e => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:59.045383 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e3ca9-3c89-4253-8c2b-a1090ae52379 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:59.045998 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e3ca9-3c89-4253-8c2b-a1090ae52379 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:59.046396 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e3ca9-3c89-4253-8c2b-a1090ae52379 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:59.048166 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 613/1247] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:30:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:30:59.054698 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ba9af-229e-4746-90fc-10728e5dfa2b None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:59.055150 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ba9af-229e-4746-90fc-10728e5dfa2b None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:59.055255 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ba9af-229e-4746-90fc-10728e5dfa2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:59.061342 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:59.076263 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-8d0ba9af-229e-4746-90fc-10728e5dfa2b None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:30:59.077705 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-8d0ba9af-229e-4746-90fc-10728e5dfa2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:30:59.163500 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 635/1248] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:30:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:30:59.191662 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10e3ca9-3c89-4253-8c2b-a1090ae52379 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:59.215836 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:59.329345 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10e3ca9-3c89-4253-8c2b-a1090ae52379 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:59.381094 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1bd7f7-bdf7-498b-bdb6-21fdeb02017b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:59.381975 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1bd7f7-bdf7-498b-bdb6-21fdeb02017b None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:59.382369 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1bd7f7-bdf7-498b-bdb6-21fdeb02017b None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:59.382574 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1bd7f7-bdf7-498b-bdb6-21fdeb02017b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:59.400101 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1bd7f7-bdf7-498b-bdb6-21fdeb02017b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:30:59.408224 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-dc1bd7f7-bdf7-498b-bdb6-21fdeb02017b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:30:59.408224 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:30:59.408224 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:30:59.408224 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:30:59.408224 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:30:59.408224 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:30:59.408224 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:30:59.408224 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:30:59.408224 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:30:59.408224 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:30:59.408224 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:30:59.408224 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:30:59.408224 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:30:59.408224 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:30:59.408224 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:30:59.408224 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:30:59.409369 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:30:59.409369 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:30:59.409369 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:30:59.409369 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:30:59.409369 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:30:59.409369 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:30:59.409369 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:30:59.409369 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:30:59.409369 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:30:59.409369 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:30:59.409369 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:30:59.409369 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:30:59.409369 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:30:59.409369 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:30:59.409369 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:30:59.409369 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:30:59.410508 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:30:59.410508 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:30:59.410508 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:30:59.410508 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:30:59.410508 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:30:59.410508 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:30:59.410508 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:30:59.410508 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:30:59.410508 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:30:59.410508 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:30:59.410508 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:30:59.410508 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:30:59.410508 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 614/1249] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Jun 24 08:30:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:30:59.416191 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0ba9af-229e-4746-90fc-10728e5dfa2b None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:59.538778 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0ba9af-229e-4746-90fc-10728e5dfa2b None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:59.588693 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:59.595766 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5386e83-3267-41a5-aa8f-312d0914189f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:59.596765 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5386e83-3267-41a5-aa8f-312d0914189f None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:59.597256 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5386e83-3267-41a5-aa8f-312d0914189f None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:59.597851 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5386e83-3267-41a5-aa8f-312d0914189f None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:59.598904 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5386e83-3267-41a5-aa8f-312d0914189f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:59.599364 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5386e83-3267-41a5-aa8f-312d0914189f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:59.601380 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5386e83-3267-41a5-aa8f-312d0914189f None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:59.620642 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 636/1250] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:30:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:59.621908 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1bd7f7-bdf7-498b-bdb6-21fdeb02017b None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:59.718222 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:59.751230 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1bd7f7-bdf7-498b-bdb6-21fdeb02017b None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:59.798714 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7744a28-09f0-49d1-8185-55d8e856f10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:59.799996 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7744a28-09f0-49d1-8185-55d8e856f10a None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:59.800297 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7744a28-09f0-49d1-8185-55d8e856f10a None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:59.800764 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7744a28-09f0-49d1-8185-55d8e856f10a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:59.814255 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7744a28-09f0-49d1-8185-55d8e856f10a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:59.816080 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7744a28-09f0-49d1-8185-55d8e856f10a None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:59.842812 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-a7744a28-09f0-49d1-8185-55d8e856f10a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:30:59.842812 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:30:59.842812 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:30:59.842812 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:30:59.842812 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:30:59.842812 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:30:59.842812 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:30:59.842812 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:30:59.842812 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:30:59.842812 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:30:59.842812 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:30:59.842812 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:30:59.842812 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:30:59.842812 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:30:59.842812 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:30:59.843906 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:30:59.843906 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:30:59.843906 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:30:59.843906 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:30:59.843906 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:30:59.843906 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:30:59.843906 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:30:59.843906 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:30:59.843906 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:30:59.843906 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:30:59.843906 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:30:59.843906 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:30:59.843906 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:30:59.843906 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:30:59.843906 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:30:59.843906 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:30:59.843906 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:30:59.844999 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:30:59.844999 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:30:59.844999 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:30:59.844999 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:30:59.844999 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 615/1251] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:30:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:30:59.851625 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5386e83-3267-41a5-aa8f-312d0914189f None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:59.990675 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5386e83-3267-41a5-aa8f-312d0914189f None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:00.035297 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48455d16-bc67-4935-88f2-b1cb1d61ed5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:00.035962 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48455d16-bc67-4935-88f2-b1cb1d61ed5f None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:00.037027 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48455d16-bc67-4935-88f2-b1cb1d61ed5f None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:00.037027 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48455d16-bc67-4935-88f2-b1cb1d61ed5f None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:00.037221 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48455d16-bc67-4935-88f2-b1cb1d61ed5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:31:00.037517 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48455d16-bc67-4935-88f2-b1cb1d61ed5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:00.038998 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48455d16-bc67-4935-88f2-b1cb1d61ed5f None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:00.054237 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 637/1252] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:30:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:31:00.060299 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7744a28-09f0-49d1-8185-55d8e856f10a None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:00.189874 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7744a28-09f0-49d1-8185-55d8e856f10a None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:00.220461 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:31:00.231767 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf440ca-7d16-4034-9bfd-67d864864134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:00.232800 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf440ca-7d16-4034-9bfd-67d864864134 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:00.233109 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf440ca-7d16-4034-9bfd-67d864864134 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:00.233433 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf440ca-7d16-4034-9bfd-67d864864134 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:00.242753 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf440ca-7d16-4034-9bfd-67d864864134 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:00.244494 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf440ca-7d16-4034-9bfd-67d864864134 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:00.263516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-ecf440ca-7d16-4034-9bfd-67d864864134 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 08:31:00.263516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:31:00.263516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:31:00.263516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:31:00.263516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:31:00.263516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:31:00.263516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:31:00.263516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:31:00.263516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:31:00.263516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:31:00.263516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:31:00.263516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:31:00.263516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:31:00.263516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:31:00.263516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:31:00.263516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:31:00.264796 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:31:00.264796 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:31:00.264796 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:31:00.264796 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:31:00.264796 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:31:00.264796 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:31:00.264796 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:31:00.264796 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:31:00.264796 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:31:00.264796 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:31:00.264796 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:31:00.264796 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:31:00.264796 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:31:00.264796 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:31:00.264796 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:31:00.264796 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:31:00.264796 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:31:00.264796 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:31:00.266208 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:31:00.266208 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:31:00.266208 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:31:00.266208 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:31:00.266208 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:31:00.266208 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:31:00.266208 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:31:00.266208 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:31:00.266208 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 08:31:00.266208 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:31:00.266208 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 616/1253] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:31:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:31:00.271159 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48455d16-bc67-4935-88f2-b1cb1d61ed5f None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:00.340717 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:31:00.408399 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48455d16-bc67-4935-88f2-b1cb1d61ed5f None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:00.454073 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da541353-c4a2-481e-a7c4-fc8ee33c5124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:00.454677 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da541353-c4a2-481e-a7c4-fc8ee33c5124 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:00.454959 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da541353-c4a2-481e-a7c4-fc8ee33c5124 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:00.455228 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da541353-c4a2-481e-a7c4-fc8ee33c5124 None admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:00.456063 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da541353-c4a2-481e-a7c4-fc8ee33c5124 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:31:00.456369 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da541353-c4a2-481e-a7c4-fc8ee33c5124 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:00.457862 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da541353-c4a2-481e-a7c4-fc8ee33c5124 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:00.478669 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 638/1254] 10.222.0.13 () {56 vars in 1143 bytes} [Fri Jun 24 08:31:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:31:00.485445 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf440ca-7d16-4034-9bfd-67d864864134 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:00.620626 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf440ca-7d16-4034-9bfd-67d864864134 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:00.667199 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7369f09d-7f8c-4e55-a7b9-3ca6afae18a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:00.667928 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369f09d-7f8c-4e55-a7b9-3ca6afae18a9 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:00.668260 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369f09d-7f8c-4e55-a7b9-3ca6afae18a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:00.668536 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369f09d-7f8c-4e55-a7b9-3ca6afae18a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:00.739384 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:31:00.760588 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7369f09d-7f8c-4e55-a7b9-3ca6afae18a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:00.763477 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7369f09d-7f8c-4e55-a7b9-3ca6afae18a9 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:00.882045 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 617/1255] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:31:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 08:31:00.883290 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da541353-c4a2-481e-a7c4-fc8ee33c5124 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:01.018379 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da541353-c4a2-481e-a7c4-fc8ee33c5124 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:01.036329 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:31:01.074711 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307acad9-f9b5-4516-906f-c0ab3d53203b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:01.078757 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307acad9-f9b5-4516-906f-c0ab3d53203b None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:01.079731 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307acad9-f9b5-4516-906f-c0ab3d53203b None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:01.079731 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307acad9-f9b5-4516-906f-c0ab3d53203b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:01.094398 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307acad9-f9b5-4516-906f-c0ab3d53203b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:01.096634 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307acad9-f9b5-4516-906f-c0ab3d53203b None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:01.117713 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-307acad9-f9b5-4516-906f-c0ab3d53203b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:31:01.117713 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:31:01.117713 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:31:01.117713 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:31:01.117713 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:31:01.117713 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:31:01.117713 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:31:01.117713 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:31:01.117713 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:31:01.117713 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:31:01.117713 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:31:01.117713 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:31:01.117713 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:31:01.117713 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:31:01.117713 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:31:01.119488 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:31:01.119488 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:31:01.119488 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:31:01.119488 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:31:01.119488 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:31:01.119488 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:31:01.119488 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:31:01.119488 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:31:01.119488 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:31:01.119488 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:31:01.119488 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:31:01.119488 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:31:01.119488 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:31:01.119488 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:31:01.119488 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:31:01.119488 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:31:01.119488 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:31:01.120690 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:31:01.120690 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:31:01.120690 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:31:01.120690 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:31:01.120690 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 639/1256] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:31:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:31:01.125678 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7369f09d-7f8c-4e55-a7b9-3ca6afae18a9 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:01.226498 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:31:01.239755 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7369f09d-7f8c-4e55-a7b9-3ca6afae18a9 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:01.276178 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05773ef4-e781-4e58-94cd-df08debba9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:01.276789 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05773ef4-e781-4e58-94cd-df08debba9cb None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:01.277149 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05773ef4-e781-4e58-94cd-df08debba9cb None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:01.277430 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05773ef4-e781-4e58-94cd-df08debba9cb None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:01.278181 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05773ef4-e781-4e58-94cd-df08debba9cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:31:01.278488 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05773ef4-e781-4e58-94cd-df08debba9cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:01.279850 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05773ef4-e781-4e58-94cd-df08debba9cb None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:01.296764 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 618/1257] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:31:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:31:01.298164 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307acad9-f9b5-4516-906f-c0ab3d53203b None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:01.427344 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307acad9-f9b5-4516-906f-c0ab3d53203b None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:01.466678 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca9c29f-7101-4ce6-b7e0-7cbb6f3a70c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:01.467510 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca9c29f-7101-4ce6-b7e0-7cbb6f3a70c8 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:01.467809 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca9c29f-7101-4ce6-b7e0-7cbb6f3a70c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:01.468091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca9c29f-7101-4ce6-b7e0-7cbb6f3a70c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:01.476067 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca9c29f-7101-4ce6-b7e0-7cbb6f3a70c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:01.477574 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca9c29f-7101-4ce6-b7e0-7cbb6f3a70c8 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:01.493113 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-4ca9c29f-7101-4ce6-b7e0-7cbb6f3a70c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:31:01.493113 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:31:01.493113 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:31:01.493113 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:31:01.493113 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:31:01.493113 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:31:01.493113 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:31:01.493113 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:31:01.493113 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:31:01.493113 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:31:01.493113 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:31:01.493113 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:31:01.493113 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:31:01.493113 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:31:01.493113 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:31:01.495134 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:31:01.495134 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:31:01.495134 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:31:01.495134 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:31:01.495134 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:31:01.495134 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:31:01.495134 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:31:01.495134 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:31:01.495134 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:31:01.495134 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:31:01.495134 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:31:01.495134 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:31:01.495134 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:31:01.495134 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:31:01.495134 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:31:01.495134 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:31:01.495134 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:31:01.496213 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:31:01.496213 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:31:01.496213 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:31:01.496213 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:31:01.496213 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:31:01.496213 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:31:01.496213 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:31:01.496213 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:31:01.496923 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 640/1258] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:31:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:31:01.501248 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05773ef4-e781-4e58-94cd-df08debba9cb None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:01.608019 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05773ef4-e781-4e58-94cd-df08debba9cb None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:01.643608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460ea394-9623-4b72-ac62-359fb43cbacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:01.644270 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460ea394-9623-4b72-ac62-359fb43cbacc None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:01.644551 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460ea394-9623-4b72-ac62-359fb43cbacc None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:01.644825 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460ea394-9623-4b72-ac62-359fb43cbacc None admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:01.645609 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460ea394-9623-4b72-ac62-359fb43cbacc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:31:01.645944 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460ea394-9623-4b72-ac62-359fb43cbacc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:01.647383 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460ea394-9623-4b72-ac62-359fb43cbacc None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:01.660734 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 619/1259] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Jun 24 08:31:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:31:01.667434 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca9c29f-7101-4ce6-b7e0-7cbb6f3a70c8 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:01.695366 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:31:01.776768 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca9c29f-7101-4ce6-b7e0-7cbb6f3a70c8 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:01.812398 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2523d8d9-24f3-42b3-b9e0-df14e8641529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:01.813063 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2523d8d9-24f3-42b3-b9e0-df14e8641529 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:01.813345 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2523d8d9-24f3-42b3-b9e0-df14e8641529 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:01.813616 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2523d8d9-24f3-42b3-b9e0-df14e8641529 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:01.814481 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2523d8d9-24f3-42b3-b9e0-df14e8641529 None admin] RBAC: Adding query filter params (role.id=acfe707eee6944ad94d1915fc9ee7ea3, scope.project.id=180b0af5863349e99ae412b07c1e9465, user.id=39cc268bad604dd08594e3b39c7ecf5a) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:31:01.814790 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2523d8d9-24f3-42b3-b9e0-df14e8641529 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:01.816231 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2523d8d9-24f3-42b3-b9e0-df14e8641529 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:01.822421 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 641/1260] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Jun 24 08:31:01 2022] GET /identity/v3/role_assignments?user.id=39cc268bad604dd08594e3b39c7ecf5a&scope.project.id=180b0af5863349e99ae412b07c1e9465&role.id=acfe707eee6944ad94d1915fc9ee7ea3 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:31:03.834806 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bd35b8-d833-4431-baf9-3b5896a86313 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:03.835377 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bd35b8-d833-4431-baf9-3b5896a86313 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:03.835377 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bd35b8-d833-4431-baf9-3b5896a86313 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:03.836714 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 620/1261] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:31:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:31:03.841896 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb15088-86c4-4594-b7be-faa479b69369 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:03.842289 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb15088-86c4-4594-b7be-faa479b69369 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:03.842453 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb15088-86c4-4594-b7be-faa479b69369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:03.859392 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-8bb15088-86c4-4594-b7be-faa479b69369 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:31:03.860577 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-8bb15088-86c4-4594-b7be-faa479b69369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:31:03.929038 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 642/1262] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:31:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:31:03.962068 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bd35b8-d833-4431-baf9-3b5896a86313 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:03.974411 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:31:04.068621 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bd35b8-d833-4431-baf9-3b5896a86313 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:04.101924 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9683d009-91d4-4562-a705-5403b65d6693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:04.102637 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9683d009-91d4-4562-a705-5403b65d6693 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:04.102918 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9683d009-91d4-4562-a705-5403b65d6693 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:04.103215 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9683d009-91d4-4562-a705-5403b65d6693 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:04.112780 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9683d009-91d4-4562-a705-5403b65d6693 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:31:04.120761 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-9683d009-91d4-4562-a705-5403b65d6693 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:31:04.120761 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:31:04.120761 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:31:04.120761 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:31:04.120761 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:31:04.120761 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:31:04.120761 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:31:04.120761 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:31:04.120761 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:31:04.120761 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:31:04.120761 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:31:04.120761 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:31:04.120761 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:31:04.120761 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:31:04.120761 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:31:04.120761 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:31:04.121911 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:31:04.121911 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:31:04.121911 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:31:04.121911 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:31:04.121911 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:31:04.121911 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:31:04.121911 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:31:04.121911 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:31:04.121911 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:31:04.121911 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:31:04.121911 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:31:04.121911 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:31:04.121911 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:31:04.121911 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:31:04.121911 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:31:04.121911 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:31:04.122965 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:31:04.122965 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:31:04.122965 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:31:04.122965 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:31:04.122965 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:31:04.122965 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:31:04.122965 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:31:04.122965 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:31:04.122965 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:31:04.122965 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:31:04.122965 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:31:04.122965 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:31:04.122965 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 621/1263] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Jun 24 08:31:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:31:04.128069 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb15088-86c4-4594-b7be-faa479b69369 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:04.146716 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:31:04.225896 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb15088-86c4-4594-b7be-faa479b69369 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:04.259294 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d4a11-18c8-42cc-906d-7501b86e505f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:04.259970 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d4a11-18c8-42cc-906d-7501b86e505f None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:04.260248 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d4a11-18c8-42cc-906d-7501b86e505f None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:04.260521 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d4a11-18c8-42cc-906d-7501b86e505f None admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:04.261251 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692d4a11-18c8-42cc-906d-7501b86e505f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:31:04.261552 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692d4a11-18c8-42cc-906d-7501b86e505f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:04.262935 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692d4a11-18c8-42cc-906d-7501b86e505f None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:04.270694 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 643/1264] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:31:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:31:04.276438 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9683d009-91d4-4562-a705-5403b65d6693 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:04.343651 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:31:04.378503 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9683d009-91d4-4562-a705-5403b65d6693 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:04.411587 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e9fed9-e449-4fe9-bc45-be77184cb49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:04.412359 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9fed9-e449-4fe9-bc45-be77184cb49e None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:04.412643 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9fed9-e449-4fe9-bc45-be77184cb49e None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:04.412916 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9fed9-e449-4fe9-bc45-be77184cb49e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:04.423360 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e9fed9-e449-4fe9-bc45-be77184cb49e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:04.424866 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e9fed9-e449-4fe9-bc45-be77184cb49e None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:04.447224 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-70e9fed9-e449-4fe9-bc45-be77184cb49e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:31:04.447224 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:31:04.447224 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:31:04.447224 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:31:04.447224 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:31:04.447224 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:31:04.447224 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:31:04.447224 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:31:04.447224 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:31:04.447224 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:31:04.447224 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:31:04.447224 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:31:04.447224 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:31:04.447224 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:31:04.447224 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:31:04.448504 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:31:04.448504 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:31:04.448504 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:31:04.448504 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:31:04.448504 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:31:04.448504 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:31:04.448504 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:31:04.448504 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:31:04.448504 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:31:04.448504 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:31:04.448504 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:31:04.448504 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:31:04.448504 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:31:04.448504 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:31:04.448504 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:31:04.448504 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:31:04.448504 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:31:04.449449 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:31:04.449449 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:31:04.449449 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:31:04.449449 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:31:04.449449 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 622/1265] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:31:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:31:04.454760 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d4a11-18c8-42cc-906d-7501b86e505f None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:04.535611 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:31:04.556250 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d4a11-18c8-42cc-906d-7501b86e505f None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:04.590294 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9425de0-e8e3-4658-9a4e-7881e15ca981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:04.590951 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9425de0-e8e3-4658-9a4e-7881e15ca981 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:04.591238 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9425de0-e8e3-4658-9a4e-7881e15ca981 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:04.591515 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9425de0-e8e3-4658-9a4e-7881e15ca981 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:04.592230 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9425de0-e8e3-4658-9a4e-7881e15ca981 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:31:04.592538 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9425de0-e8e3-4658-9a4e-7881e15ca981 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:04.593940 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9425de0-e8e3-4658-9a4e-7881e15ca981 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:04.606326 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 644/1266] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:31:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:31:04.611935 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e9fed9-e449-4fe9-bc45-be77184cb49e None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:04.707689 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e9fed9-e449-4fe9-bc45-be77184cb49e None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:04.727176 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:31:04.742384 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dad7c6-9606-426e-a8b1-2fb192ceb0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:04.743162 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dad7c6-9606-426e-a8b1-2fb192ceb0e2 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:04.743442 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dad7c6-9606-426e-a8b1-2fb192ceb0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:04.743717 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dad7c6-9606-426e-a8b1-2fb192ceb0e2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:04.750660 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dad7c6-9606-426e-a8b1-2fb192ceb0e2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:04.752085 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dad7c6-9606-426e-a8b1-2fb192ceb0e2 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:04.766685 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-19dad7c6-9606-426e-a8b1-2fb192ceb0e2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 08:31:04.766685 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:31:04.766685 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:31:04.766685 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:31:04.766685 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:31:04.766685 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:31:04.766685 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:31:04.766685 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:31:04.766685 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:31:04.766685 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:31:04.766685 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:31:04.766685 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:31:04.766685 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:31:04.766685 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:31:04.766685 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:31:04.766685 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:31:04.767808 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:31:04.767808 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:31:04.767808 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:31:04.767808 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:31:04.767808 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:31:04.767808 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:31:04.767808 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:31:04.767808 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:31:04.767808 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:31:04.767808 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:31:04.767808 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:31:04.767808 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:31:04.767808 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:31:04.767808 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:31:04.767808 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:31:04.767808 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:31:04.767808 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:31:04.767808 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:31:04.769208 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:31:04.769208 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:31:04.769208 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:31:04.769208 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:31:04.769208 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:31:04.769208 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:31:04.769208 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:31:04.769208 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:31:04.769208 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 08:31:04.769208 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:31:04.769208 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 623/1267] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:31:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:31:04.773536 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9425de0-e8e3-4658-9a4e-7881e15ca981 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:04.874797 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9425de0-e8e3-4658-9a4e-7881e15ca981 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:04.908724 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329ceaf0-62b1-401a-a602-bc9b92076b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:04.909382 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ceaf0-62b1-401a-a602-bc9b92076b27 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:04.909666 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ceaf0-62b1-401a-a602-bc9b92076b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:04.909975 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ceaf0-62b1-401a-a602-bc9b92076b27 None admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:04.910806 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329ceaf0-62b1-401a-a602-bc9b92076b27 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:31:04.911112 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329ceaf0-62b1-401a-a602-bc9b92076b27 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:04.912898 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329ceaf0-62b1-401a-a602-bc9b92076b27 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:04.924535 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:31:04.934462 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 645/1268] 10.222.0.13 () {56 vars in 1143 bytes} [Fri Jun 24 08:31:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:31:04.941585 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dad7c6-9606-426e-a8b1-2fb192ceb0e2 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:05.048237 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dad7c6-9606-426e-a8b1-2fb192ceb0e2 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:05.081703 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d428ef0-ffe9-491c-bd5e-ce1dd0a1867d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:05.082503 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d428ef0-ffe9-491c-bd5e-ce1dd0a1867d None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:05.082788 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d428ef0-ffe9-491c-bd5e-ce1dd0a1867d None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:05.083063 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d428ef0-ffe9-491c-bd5e-ce1dd0a1867d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:05.093787 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d428ef0-ffe9-491c-bd5e-ce1dd0a1867d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:05.095231 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d428ef0-ffe9-491c-bd5e-ce1dd0a1867d None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:05.112397 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-0d428ef0-ffe9-491c-bd5e-ce1dd0a1867d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:31:05.112397 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:31:05.112397 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:31:05.112397 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:31:05.112397 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:31:05.112397 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:31:05.112397 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:31:05.112397 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:31:05.112397 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:31:05.112397 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:31:05.112397 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:31:05.112397 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:31:05.112397 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:31:05.112397 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:31:05.112397 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:31:05.113434 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:31:05.113434 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:31:05.113434 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:31:05.113434 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:31:05.113434 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:31:05.113434 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:31:05.113434 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:31:05.113434 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:31:05.113434 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:31:05.113434 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:31:05.113434 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:31:05.113434 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:31:05.113434 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:31:05.113434 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:31:05.113434 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:31:05.113434 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:31:05.113434 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:31:05.114474 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:31:05.114474 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:31:05.114474 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:31:05.114474 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:31:05.114474 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 624/1269] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:31:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:31:05.119518 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329ceaf0-62b1-401a-a602-bc9b92076b27 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:05.221765 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329ceaf0-62b1-401a-a602-bc9b92076b27 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:05.254276 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05afa99-f96b-4dce-a5e4-5179a777e8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:05.254923 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05afa99-f96b-4dce-a5e4-5179a777e8f5 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:05.255208 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05afa99-f96b-4dce-a5e4-5179a777e8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:05.255483 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05afa99-f96b-4dce-a5e4-5179a777e8f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:05.256193 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05afa99-f96b-4dce-a5e4-5179a777e8f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:31:05.256503 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05afa99-f96b-4dce-a5e4-5179a777e8f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:05.257894 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05afa99-f96b-4dce-a5e4-5179a777e8f5 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:05.269707 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 646/1270] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:31:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:31:05.275356 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d428ef0-ffe9-491c-bd5e-ce1dd0a1867d None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:05.306822 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:31:05.380349 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d428ef0-ffe9-491c-bd5e-ce1dd0a1867d None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:05.414649 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a9a8fe-7abf-4ace-bcf4-7ccc3443d53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:05.415427 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a9a8fe-7abf-4ace-bcf4-7ccc3443d53a None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:05.415851 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a9a8fe-7abf-4ace-bcf4-7ccc3443d53a None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:05.417133 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a9a8fe-7abf-4ace-bcf4-7ccc3443d53a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:05.423596 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a9a8fe-7abf-4ace-bcf4-7ccc3443d53a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:05.425049 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a9a8fe-7abf-4ace-bcf4-7ccc3443d53a None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:05.438827 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-24a9a8fe-7abf-4ace-bcf4-7ccc3443d53a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:31:05.438827 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:31:05.438827 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:31:05.438827 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:31:05.438827 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:31:05.438827 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:31:05.438827 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:31:05.438827 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:31:05.438827 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:31:05.438827 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:31:05.438827 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:31:05.438827 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:31:05.438827 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:31:05.438827 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:31:05.438827 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:31:05.439836 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:31:05.439836 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:31:05.439836 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:31:05.439836 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:31:05.439836 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:31:05.439836 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:31:05.439836 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:31:05.439836 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:31:05.439836 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:31:05.439836 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:31:05.439836 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:31:05.439836 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:31:05.439836 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:31:05.439836 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:31:05.439836 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:31:05.439836 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:31:05.439836 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:31:05.445763 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:31:05.445763 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:31:05.445763 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:31:05.445763 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:31:05.445763 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:31:05.445763 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:31:05.445763 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:31:05.445763 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:31:05.445763 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 625/1271] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:31:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:31:05.446362 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05afa99-f96b-4dce-a5e4-5179a777e8f5 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:05.504469 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:31:05.547354 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05afa99-f96b-4dce-a5e4-5179a777e8f5 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:05.580586 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3477138-6883-4262-a7f6-ebd0610f7140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:05.581170 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3477138-6883-4262-a7f6-ebd0610f7140 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:05.581450 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3477138-6883-4262-a7f6-ebd0610f7140 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:05.581748 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3477138-6883-4262-a7f6-ebd0610f7140 None admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:05.582519 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3477138-6883-4262-a7f6-ebd0610f7140 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:31:05.582822 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3477138-6883-4262-a7f6-ebd0610f7140 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:05.584232 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3477138-6883-4262-a7f6-ebd0610f7140 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:05.596415 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 647/1272] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Jun 24 08:31:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:31:05.602588 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a9a8fe-7abf-4ace-bcf4-7ccc3443d53a None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:05.692711 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:31:05.701789 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a9a8fe-7abf-4ace-bcf4-7ccc3443d53a None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:05.734598 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4251b0-229f-4ba3-8525-c911a75eb6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:05.735572 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4251b0-229f-4ba3-8525-c911a75eb6a5 None admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:05.735861 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4251b0-229f-4ba3-8525-c911a75eb6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:05.736145 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4251b0-229f-4ba3-8525-c911a75eb6a5 None admin] PATH_INFO: `/v3/projects/180b0af5863349e99ae412b07c1e9465/users/39cc268bad604dd08594e3b39c7ecf5a/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:05.738354 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4251b0-229f-4ba3-8525-c911a75eb6a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=180b0af5863349e99ae412b07c1e9465, user_id=39cc268bad604dd08594e3b39c7ecf5a, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:05.739964 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4251b0-229f-4ba3-8525-c911a75eb6a5 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:05.752931 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 626/1273] 10.222.0.13 () {58 vars in 1439 bytes} [Fri Jun 24 08:31:05 2022] PUT /identity/v3/projects/180b0af5863349e99ae412b07c1e9465/users/39cc268bad604dd08594e3b39c7ecf5a/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:31:07.775894 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb251cc-8fb6-4cea-8fbe-e7b9350fc6e8 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:07.776442 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb251cc-8fb6-4cea-8fbe-e7b9350fc6e8 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:07.776442 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb251cc-8fb6-4cea-8fbe-e7b9350fc6e8 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:07.777807 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 648/1274] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:31:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:31:07.782955 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b8ec9e-c225-4b0c-a345-9d8c9fd80d1c None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:07.783362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b8ec9e-c225-4b0c-a345-9d8c9fd80d1c None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:07.783522 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b8ec9e-c225-4b0c-a345-9d8c9fd80d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:07.799802 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-66b8ec9e-c225-4b0c-a345-9d8c9fd80d1c None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:31:07.801004 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-66b8ec9e-c225-4b0c-a345-9d8c9fd80d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:31:07.865852 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 627/1275] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:31:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:31:07.889908 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb251cc-8fb6-4cea-8fbe-e7b9350fc6e8 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:08.007903 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb251cc-8fb6-4cea-8fbe-e7b9350fc6e8 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:08.025507 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:31:08.042970 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ab406-c092-4cfc-9e67-6f9e59d906d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:08.043689 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ab406-c092-4cfc-9e67-6f9e59d906d6 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:08.043976 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ab406-c092-4cfc-9e67-6f9e59d906d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:08.044253 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ab406-c092-4cfc-9e67-6f9e59d906d6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:08.054448 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75ab406-c092-4cfc-9e67-6f9e59d906d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:31:08.062546 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-d75ab406-c092-4cfc-9e67-6f9e59d906d6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:31:08.062546 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:31:08.062546 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:31:08.062546 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:31:08.062546 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:31:08.062546 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:31:08.062546 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:31:08.062546 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:31:08.062546 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:31:08.062546 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:31:08.062546 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:31:08.062546 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:31:08.062546 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:31:08.062546 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:31:08.062546 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:31:08.062546 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:31:08.063632 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:31:08.063632 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:31:08.063632 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:31:08.063632 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:31:08.063632 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:31:08.063632 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:31:08.063632 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:31:08.063632 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:31:08.063632 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:31:08.063632 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:31:08.063632 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:31:08.063632 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:31:08.063632 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:31:08.063632 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:31:08.063632 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:31:08.063632 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:31:08.064669 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:31:08.064669 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:31:08.064669 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:31:08.064669 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:31:08.064669 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:31:08.064669 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:31:08.064669 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:31:08.064669 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:31:08.064669 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:31:08.064669 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:31:08.064669 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:31:08.064669 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:31:08.064669 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 649/1276] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Jun 24 08:31:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:31:08.069761 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b8ec9e-c225-4b0c-a345-9d8c9fd80d1c None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:08.148574 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:31:08.168349 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b8ec9e-c225-4b0c-a345-9d8c9fd80d1c None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:08.201242 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1527d6-9a21-4335-9a97-5f32be47322d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:08.201926 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1527d6-9a21-4335-9a97-5f32be47322d None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:08.202214 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1527d6-9a21-4335-9a97-5f32be47322d None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:08.202492 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1527d6-9a21-4335-9a97-5f32be47322d None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:08.203225 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1527d6-9a21-4335-9a97-5f32be47322d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:31:08.203531 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1527d6-9a21-4335-9a97-5f32be47322d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:08.204907 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1527d6-9a21-4335-9a97-5f32be47322d None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:08.212951 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 628/1277] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Jun 24 08:31:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:31:08.218396 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ab406-c092-4cfc-9e67-6f9e59d906d6 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:08.315619 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ab406-c092-4cfc-9e67-6f9e59d906d6 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:08.347762 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f27f616-4895-4aec-8c2a-3a0fae7c0be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:08.348531 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f27f616-4895-4aec-8c2a-3a0fae7c0be9 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:08.348814 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f27f616-4895-4aec-8c2a-3a0fae7c0be9 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:08.349088 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f27f616-4895-4aec-8c2a-3a0fae7c0be9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:08.360254 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f27f616-4895-4aec-8c2a-3a0fae7c0be9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:08.361713 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f27f616-4895-4aec-8c2a-3a0fae7c0be9 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:08.378886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-2f27f616-4895-4aec-8c2a-3a0fae7c0be9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:31:08.378886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:31:08.378886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:31:08.378886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:31:08.378886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:31:08.378886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:31:08.378886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:31:08.378886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:31:08.378886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:31:08.378886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:31:08.378886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:31:08.378886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:31:08.378886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:31:08.378886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:31:08.378886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:31:08.379934 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:31:08.379934 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:31:08.379934 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:31:08.379934 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:31:08.379934 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:31:08.379934 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:31:08.379934 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:31:08.379934 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:31:08.379934 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:31:08.379934 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:31:08.379934 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:31:08.379934 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:31:08.379934 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:31:08.379934 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:31:08.379934 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:31:08.379934 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:31:08.379934 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:31:08.380890 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:31:08.380890 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:31:08.380890 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:31:08.380890 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:31:08.380890 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 650/1278] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:31:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:31:08.385928 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1527d6-9a21-4335-9a97-5f32be47322d None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:08.481937 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1527d6-9a21-4335-9a97-5f32be47322d None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:08.515414 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a615d2-6300-4ca3-964b-4266cb5505b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:08.516065 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a615d2-6300-4ca3-964b-4266cb5505b4 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:08.516347 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a615d2-6300-4ca3-964b-4266cb5505b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:08.516623 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a615d2-6300-4ca3-964b-4266cb5505b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:08.517327 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a615d2-6300-4ca3-964b-4266cb5505b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:31:08.517630 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a615d2-6300-4ca3-964b-4266cb5505b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:08.519011 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a615d2-6300-4ca3-964b-4266cb5505b4 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:08.530868 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 629/1279] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:31:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:31:08.536790 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f27f616-4895-4aec-8c2a-3a0fae7c0be9 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:08.568819 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:31:08.637549 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f27f616-4895-4aec-8c2a-3a0fae7c0be9 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:08.671173 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a17c97-eee7-4f9d-8930-0b2ef7ab1606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:08.671950 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a17c97-eee7-4f9d-8930-0b2ef7ab1606 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:08.672235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a17c97-eee7-4f9d-8930-0b2ef7ab1606 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:08.672512 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a17c97-eee7-4f9d-8930-0b2ef7ab1606 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:08.679798 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a17c97-eee7-4f9d-8930-0b2ef7ab1606 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:08.681224 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a17c97-eee7-4f9d-8930-0b2ef7ab1606 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:08.695351 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-f5a17c97-eee7-4f9d-8930-0b2ef7ab1606 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 08:31:08.695351 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:31:08.695351 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:31:08.695351 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:31:08.695351 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:31:08.695351 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:31:08.695351 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:31:08.695351 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:31:08.695351 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:31:08.695351 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:31:08.695351 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:31:08.695351 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:31:08.695351 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:31:08.695351 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:31:08.695351 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:31:08.695351 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:31:08.696403 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:31:08.696403 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:31:08.696403 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:31:08.696403 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:31:08.696403 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:31:08.696403 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:31:08.696403 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:31:08.696403 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:31:08.696403 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:31:08.696403 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:31:08.696403 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:31:08.696403 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:31:08.696403 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:31:08.696403 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:31:08.696403 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:31:08.696403 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:31:08.696403 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:31:08.696403 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:31:08.697675 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:31:08.697675 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:31:08.697675 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:31:08.697675 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:31:08.697675 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:31:08.697675 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:31:08.697675 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:31:08.697675 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:31:08.697675 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 08:31:08.697675 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:31:08.697675 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 651/1280] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:31:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:31:08.702641 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a615d2-6300-4ca3-964b-4266cb5505b4 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:08.706403 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:31:08.802790 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a615d2-6300-4ca3-964b-4266cb5505b4 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:08.836319 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02521365-b1ae-4652-b63c-e2d41e675d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:08.836904 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02521365-b1ae-4652-b63c-e2d41e675d91 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:08.837187 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02521365-b1ae-4652-b63c-e2d41e675d91 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:08.837463 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02521365-b1ae-4652-b63c-e2d41e675d91 None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:08.839237 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02521365-b1ae-4652-b63c-e2d41e675d91 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:31:08.839237 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02521365-b1ae-4652-b63c-e2d41e675d91 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:08.840619 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02521365-b1ae-4652-b63c-e2d41e675d91 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:08.858017 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 630/1281] 10.222.0.13 () {56 vars in 1143 bytes} [Fri Jun 24 08:31:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:31:08.863649 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a17c97-eee7-4f9d-8930-0b2ef7ab1606 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:08.955930 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:31:08.967111 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a17c97-eee7-4f9d-8930-0b2ef7ab1606 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:08.999588 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec7b80f-160f-4a12-9e15-18f331c72456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:09.000222 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec7b80f-160f-4a12-9e15-18f331c72456 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:09.000503 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec7b80f-160f-4a12-9e15-18f331c72456 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:09.000774 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec7b80f-160f-4a12-9e15-18f331c72456 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:09.066870 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec7b80f-160f-4a12-9e15-18f331c72456 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:09.068298 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec7b80f-160f-4a12-9e15-18f331c72456 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:09.154151 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:31:09.164170 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 652/1282] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Jun 24 08:31:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 08:31:09.170524 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02521365-b1ae-4652-b63c-e2d41e675d91 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:09.229912 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:31:09.267188 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02521365-b1ae-4652-b63c-e2d41e675d91 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:09.300337 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcac73f-8447-42c7-9215-b589a1241f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:09.301106 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcac73f-8447-42c7-9215-b589a1241f78 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:09.301389 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcac73f-8447-42c7-9215-b589a1241f78 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:09.301660 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcac73f-8447-42c7-9215-b589a1241f78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:09.311908 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcac73f-8447-42c7-9215-b589a1241f78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:09.313331 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcac73f-8447-42c7-9215-b589a1241f78 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:09.330007 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-bdcac73f-8447-42c7-9215-b589a1241f78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:31:09.330007 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:31:09.330007 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:31:09.330007 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:31:09.330007 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:31:09.330007 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:31:09.330007 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:31:09.330007 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:31:09.330007 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:31:09.330007 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:31:09.330007 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:31:09.330007 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:31:09.330007 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:31:09.330007 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:31:09.330007 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:31:09.331149 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:31:09.331149 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:31:09.331149 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:31:09.331149 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:31:09.331149 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:31:09.331149 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:31:09.331149 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:31:09.331149 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:31:09.331149 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:31:09.331149 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:31:09.331149 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:31:09.331149 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:31:09.331149 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:31:09.331149 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:31:09.331149 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:31:09.331149 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:31:09.331149 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:31:09.332257 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:31:09.332257 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:31:09.332257 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:31:09.332257 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:31:09.332257 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 631/1283] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:31:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:31:09.336740 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec7b80f-160f-4a12-9e15-18f331c72456 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:09.436143 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec7b80f-160f-4a12-9e15-18f331c72456 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:09.469105 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552f0b01-a7f1-4215-8b17-1a12ea19d452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:09.469779 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f0b01-a7f1-4215-8b17-1a12ea19d452 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:09.470064 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f0b01-a7f1-4215-8b17-1a12ea19d452 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:09.470339 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f0b01-a7f1-4215-8b17-1a12ea19d452 None admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:09.471047 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552f0b01-a7f1-4215-8b17-1a12ea19d452 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:31:09.471351 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552f0b01-a7f1-4215-8b17-1a12ea19d452 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:09.472703 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552f0b01-a7f1-4215-8b17-1a12ea19d452 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:09.484216 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 653/1284] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Jun 24 08:31:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:31:09.489789 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcac73f-8447-42c7-9215-b589a1241f78 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:09.588539 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcac73f-8447-42c7-9215-b589a1241f78 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:09.601952 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:31:09.622225 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a662fe8-b610-40d8-b5e7-a71c36388c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:09.622999 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a662fe8-b610-40d8-b5e7-a71c36388c1b None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:09.623287 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a662fe8-b610-40d8-b5e7-a71c36388c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:09.623562 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a662fe8-b610-40d8-b5e7-a71c36388c1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:09.630023 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a662fe8-b610-40d8-b5e7-a71c36388c1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:09.631469 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a662fe8-b610-40d8-b5e7-a71c36388c1b None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:09.645103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-5a662fe8-b610-40d8-b5e7-a71c36388c1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:31:09.645103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:31:09.645103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:31:09.645103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:31:09.645103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:31:09.645103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:31:09.645103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:31:09.645103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:31:09.645103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:31:09.645103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:31:09.645103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:31:09.645103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:31:09.645103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:31:09.645103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:31:09.645103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:31:09.646174 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:31:09.646174 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:31:09.646174 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:31:09.646174 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:31:09.646174 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:31:09.646174 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:31:09.646174 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:31:09.646174 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:31:09.646174 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 24 08:31:09.646174 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:31:09.646174 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:31:09.646174 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:31:09.646174 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:31:09.646174 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:31:09.646174 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:31:09.646174 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:31:09.646174 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:31:09.647199 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:31:09.647199 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:31:09.647199 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:31:09.647199 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:31:09.647199 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:31:09.647199 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:31:09.647199 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:31:09.647199 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 24 08:31:09.647199 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 632/1285] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Jun 24 08:31:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:31:09.652814 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552f0b01-a7f1-4215-8b17-1a12ea19d452 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:09.701779 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:31:09.752026 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552f0b01-a7f1-4215-8b17-1a12ea19d452 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:09.784864 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4633d31e-96d6-411c-a968-19206d69f988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:09.785505 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633d31e-96d6-411c-a968-19206d69f988 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:09.785818 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633d31e-96d6-411c-a968-19206d69f988 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:09.786091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633d31e-96d6-411c-a968-19206d69f988 None admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:09.786860 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4633d31e-96d6-411c-a968-19206d69f988 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:31:09.787164 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4633d31e-96d6-411c-a968-19206d69f988 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:09.788571 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4633d31e-96d6-411c-a968-19206d69f988 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:09.807558 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 654/1286] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Jun 24 08:31:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:31:09.813924 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a662fe8-b610-40d8-b5e7-a71c36388c1b None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:09.915395 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a662fe8-b610-40d8-b5e7-a71c36388c1b None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:09.952220 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e1ddb-31b5-4706-8242-f09b52c369c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:09.952794 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e1ddb-31b5-4706-8242-f09b52c369c0 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:09.953082 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e1ddb-31b5-4706-8242-f09b52c369c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:09.953362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e1ddb-31b5-4706-8242-f09b52c369c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:09.954212 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2e1ddb-31b5-4706-8242-f09b52c369c0 None admin] RBAC: Adding query filter params (role.id=acfe707eee6944ad94d1915fc9ee7ea3, scope.project.id=180b0af5863349e99ae412b07c1e9465, user.id=39cc268bad604dd08594e3b39c7ecf5a) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:31:09.954526 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2e1ddb-31b5-4706-8242-f09b52c369c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:09.955961 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2e1ddb-31b5-4706-8242-f09b52c369c0 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:09.961905 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 633/1287] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Jun 24 08:31:09 2022] GET /identity/v3/role_assignments?user.id=39cc268bad604dd08594e3b39c7ecf5a&scope.project.id=180b0af5863349e99ae412b07c1e9465&role.id=acfe707eee6944ad94d1915fc9ee7ea3 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:31:12.004678 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c55e7-fe29-4a2b-bb16-98382eaa8905 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:12.005236 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c55e7-fe29-4a2b-bb16-98382eaa8905 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:12.005236 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c55e7-fe29-4a2b-bb16-98382eaa8905 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:12.006614 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 655/1288] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:31:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:31:12.012041 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37428a6-a997-4249-a2b1-bde74d1cf8e0 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:12.012436 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37428a6-a997-4249-a2b1-bde74d1cf8e0 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:12.012592 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37428a6-a997-4249-a2b1-bde74d1cf8e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:12.029521 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-c37428a6-a997-4249-a2b1-bde74d1cf8e0 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:31:12.030711 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-c37428a6-a997-4249-a2b1-bde74d1cf8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:31:12.059013 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:31:12.095830 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 634/1289] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:31:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:31:12.119670 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14c55e7-fe29-4a2b-bb16-98382eaa8905 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:12.218666 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14c55e7-fe29-4a2b-bb16-98382eaa8905 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:12.229321 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:31:12.252288 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d328f7-d457-4513-8f08-6cfc58c3f21b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:12.253076 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d328f7-d457-4513-8f08-6cfc58c3f21b None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:12.253361 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d328f7-d457-4513-8f08-6cfc58c3f21b None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:12.253637 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d328f7-d457-4513-8f08-6cfc58c3f21b None admin] PATH_INFO: `/v3/services/placement` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:12.254307 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d328f7-d457-4513-8f08-6cfc58c3f21b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:12.255715 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d328f7-d457-4513-8f08-6cfc58c3f21b None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:12.268213 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-f0d328f7-d457-4513-8f08-6cfc58c3f21b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 08:31:12.268213 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:31:12.268213 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 08:31:12.268213 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 08:31:12.268213 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 08:31:12.268213 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 08:31:12.268213 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 08:31:12.268213 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:31:12.268213 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 08:31:12.268213 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:31:12.268213 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:31:12.268213 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:31:12.268213 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:31:12.268213 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:31:12.268213 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:31:12.268213 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:31:12.268213 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:31:12.269398 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:31:12.269398 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:31:12.269398 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:31:12.269398 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:31:12.269398 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:31:12.269398 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:31:12.269398 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 08:31:12.269398 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 08:31:12.269398 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 08:31:12.269398 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 08:31:12.269398 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:31:12.269398 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:31:12.269398 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:31:12.269398 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:31:12.269398 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:31:12.269398 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:31:12.270276 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:31:12.270276 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:31:12.270276 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:31:12.270276 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:31:12.270276 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:31:12.270276 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:31:12.270276 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:31:12.270276 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:31:12.270276 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:31:12.270276 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:31:12.270276 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 08:31:12.270276 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:31:12.270276 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 08:31:12.270276 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:31:12.270276 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 656/1290] 10.222.0.13 () {56 vars in 1117 bytes} [Fri Jun 24 08:31:12 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:31:12.275861 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37428a6-a997-4249-a2b1-bde74d1cf8e0 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:12.370209 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37428a6-a997-4249-a2b1-bde74d1cf8e0 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:12.403968 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b9069-1f92-4f25-94b6-30957e615e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:12.404616 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b9069-1f92-4f25-94b6-30957e615e52 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:12.404897 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b9069-1f92-4f25-94b6-30957e615e52 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:12.405168 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b9069-1f92-4f25-94b6-30957e615e52 None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:12.405895 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1b9069-1f92-4f25-94b6-30957e615e52 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:31:12.406202 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1b9069-1f92-4f25-94b6-30957e615e52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:12.407561 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1b9069-1f92-4f25-94b6-30957e615e52 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:12.412239 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 635/1291] 10.222.0.13 () {56 vars in 1116 bytes} [Fri Jun 24 08:31:12 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:31:12.418133 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d328f7-d457-4513-8f08-6cfc58c3f21b None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:12.522708 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d328f7-d457-4513-8f08-6cfc58c3f21b None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:12.555789 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3369e373-e422-4190-a8f4-b14cae2729a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:12.557140 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3369e373-e422-4190-a8f4-b14cae2729a2 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:12.557582 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3369e373-e422-4190-a8f4-b14cae2729a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:12.557582 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3369e373-e422-4190-a8f4-b14cae2729a2 None admin] PATH_INFO: `/v3/services` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:12.558338 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3369e373-e422-4190-a8f4-b14cae2729a2 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:31:12.558648 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3369e373-e422-4190-a8f4-b14cae2729a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:12.560018 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3369e373-e422-4190-a8f4-b14cae2729a2 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:12.564391 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 657/1292] 10.222.0.13 () {56 vars in 1116 bytes} [Fri Jun 24 08:31:12 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:31:14.617982 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f8627-216d-4207-a3d3-f7328d06bd7a None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:14.618501 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f8627-216d-4207-a3d3-f7328d06bd7a None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:14.618586 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f8627-216d-4207-a3d3-f7328d06bd7a None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:14.619923 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 636/1293] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:31:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:31:14.624975 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153671db-395e-4c9e-8873-bbec009073c9 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:14.625387 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153671db-395e-4c9e-8873-bbec009073c9 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:14.625479 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153671db-395e-4c9e-8873-bbec009073c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:14.643445 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-153671db-395e-4c9e-8873-bbec009073c9 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:31:14.644614 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-153671db-395e-4c9e-8873-bbec009073c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:31:14.673116 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:31:14.709952 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 658/1294] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:31:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:31:14.733304 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5f8627-216d-4207-a3d3-f7328d06bd7a None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:14.764483 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:31:14.833053 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5f8627-216d-4207-a3d3-f7328d06bd7a None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:14.866746 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac671f86-2bfd-445f-91a1-1fc55d4ae2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:14.867393 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac671f86-2bfd-445f-91a1-1fc55d4ae2a9 None admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:14.867675 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac671f86-2bfd-445f-91a1-1fc55d4ae2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:14.867949 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac671f86-2bfd-445f-91a1-1fc55d4ae2a9 None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:14.868664 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac671f86-2bfd-445f-91a1-1fc55d4ae2a9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:14.870585 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac671f86-2bfd-445f-91a1-1fc55d4ae2a9 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:14.883593 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 637/1295] 10.222.0.13 () {60 vars in 1141 bytes} [Fri Jun 24 08:31:14 2022] POST /identity/v3/services => generated 246 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:31:16.913416 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7f89cd-9677-4027-9c35-2a36869be967 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:16.914070 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7f89cd-9677-4027-9c35-2a36869be967 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:16.914070 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7f89cd-9677-4027-9c35-2a36869be967 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:16.915602 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 659/1296] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:31:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:31:16.921494 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a5a969-6772-4d79-ad71-e552eede3826 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:16.921913 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a5a969-6772-4d79-ad71-e552eede3826 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:16.922070 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a5a969-6772-4d79-ad71-e552eede3826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:16.940709 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-05a5a969-6772-4d79-ad71-e552eede3826 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:31:16.941973 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-05a5a969-6772-4d79-ad71-e552eede3826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:31:17.014115 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 638/1297] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:31:16 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:31:17.036181 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7f89cd-9677-4027-9c35-2a36869be967 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:17.136790 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7f89cd-9677-4027-9c35-2a36869be967 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:17.170142 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c296bf35-f03a-41d8-8b70-fc91f4a68aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:17.170927 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c296bf35-f03a-41d8-8b70-fc91f4a68aa8 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:17.171212 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c296bf35-f03a-41d8-8b70-fc91f4a68aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:17.171489 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c296bf35-f03a-41d8-8b70-fc91f4a68aa8 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:17.172140 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c296bf35-f03a-41d8-8b70-fc91f4a68aa8 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:17.173502 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c296bf35-f03a-41d8-8b70-fc91f4a68aa8 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:17.185933 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-c296bf35-f03a-41d8-8b70-fc91f4a68aa8 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 08:31:17.185933 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:31:17.185933 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 08:31:17.185933 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 08:31:17.185933 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 08:31:17.185933 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 08:31:17.185933 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 08:31:17.185933 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:31:17.185933 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 08:31:17.185933 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:31:17.185933 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:31:17.185933 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:31:17.185933 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:31:17.185933 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:31:17.185933 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:31:17.185933 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:31:17.185933 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:31:17.187084 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:31:17.187084 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:31:17.187084 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:31:17.187084 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:31:17.187084 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:31:17.187084 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:31:17.187084 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 08:31:17.187084 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 08:31:17.187084 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 08:31:17.187084 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 08:31:17.187084 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:31:17.187084 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:31:17.187084 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:31:17.187084 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:31:17.187084 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:31:17.187084 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:31:17.192886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:31:17.192886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:31:17.192886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:31:17.192886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:31:17.192886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:31:17.192886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:31:17.192886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:31:17.192886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:31:17.192886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:31:17.192886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:31:17.192886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 08:31:17.192886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:31:17.192886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 08:31:17.192886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:31:17.192886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 660/1298] 10.222.0.13 () {56 vars in 1117 bytes} [Fri Jun 24 08:31:17 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:31:17.193884 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a5a969-6772-4d79-ad71-e552eede3826 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:17.212152 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:31:17.291040 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a5a969-6772-4d79-ad71-e552eede3826 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:17.325975 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269ad10b-7b6b-491e-9d08-9efe925751b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:17.326632 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ad10b-7b6b-491e-9d08-9efe925751b5 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:17.326919 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ad10b-7b6b-491e-9d08-9efe925751b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:17.327196 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ad10b-7b6b-491e-9d08-9efe925751b5 None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:17.327891 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269ad10b-7b6b-491e-9d08-9efe925751b5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:31:17.328199 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269ad10b-7b6b-491e-9d08-9efe925751b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:17.329579 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269ad10b-7b6b-491e-9d08-9efe925751b5 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:17.334610 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 639/1299] 10.222.0.13 () {56 vars in 1116 bytes} [Fri Jun 24 08:31:17 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:31:17.340807 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c296bf35-f03a-41d8-8b70-fc91f4a68aa8 None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:17.373422 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:31:17.443006 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c296bf35-f03a-41d8-8b70-fc91f4a68aa8 None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:17.476137 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305b119a-cf25-46c0-b45d-f927a50c1626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:17.476789 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305b119a-cf25-46c0-b45d-f927a50c1626 None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:17.477076 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305b119a-cf25-46c0-b45d-f927a50c1626 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:17.477355 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305b119a-cf25-46c0-b45d-f927a50c1626 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:17.478168 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305b119a-cf25-46c0-b45d-f927a50c1626 None admin] RBAC: Adding query filter params (interface=public, service_id=4d4a46a7922e4cfcbc32b42fa58f80b8, region_id=RegionOne) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:31:17.479490 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305b119a-cf25-46c0-b45d-f927a50c1626 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:17.480862 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305b119a-cf25-46c0-b45d-f927a50c1626 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:17.485828 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 661/1300] 10.222.0.13 () {56 vars in 1251 bytes} [Fri Jun 24 08:31:17 2022] GET /identity/v3/endpoints?service_id=4d4a46a7922e4cfcbc32b42fa58f80b8&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:31:17.491446 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269ad10b-7b6b-491e-9d08-9efe925751b5 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:17.570842 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:31:17.588686 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269ad10b-7b6b-491e-9d08-9efe925751b5 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:17.622272 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af107d28-39eb-48e2-99e2-603bae9728f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:17.622916 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af107d28-39eb-48e2-99e2-603bae9728f0 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:17.623202 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af107d28-39eb-48e2-99e2-603bae9728f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:17.623477 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af107d28-39eb-48e2-99e2-603bae9728f0 None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:17.624142 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af107d28-39eb-48e2-99e2-603bae9728f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:17.625509 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af107d28-39eb-48e2-99e2-603bae9728f0 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:17.632214 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 640/1301] 10.222.0.13 () {56 vars in 1087 bytes} [Fri Jun 24 08:31:17 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:31:19.620162 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4fdcec-eb4e-45c7-b204-6f30b872286b None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:19.620683 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4fdcec-eb4e-45c7-b204-6f30b872286b None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:19.620798 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4fdcec-eb4e-45c7-b204-6f30b872286b None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:19.622180 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 662/1302] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:31:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:31:19.627858 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b286dee-345a-4471-b1f8-d310522b3ea8 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:19.628239 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b286dee-345a-4471-b1f8-d310522b3ea8 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:19.628414 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b286dee-345a-4471-b1f8-d310522b3ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:19.645127 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-6b286dee-345a-4471-b1f8-d310522b3ea8 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:31:19.646324 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-6b286dee-345a-4471-b1f8-d310522b3ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:31:19.709897 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 641/1303] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:31:19 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:31:19.733384 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4fdcec-eb4e-45c7-b204-6f30b872286b None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:19.827089 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:31:19.834038 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4fdcec-eb4e-45c7-b204-6f30b872286b None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:19.873354 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103cc7b5-bde9-43cf-ac21-bd0a4c455f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:19.874106 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103cc7b5-bde9-43cf-ac21-bd0a4c455f2e None admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:19.874388 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103cc7b5-bde9-43cf-ac21-bd0a4c455f2e None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:19.874660 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103cc7b5-bde9-43cf-ac21-bd0a4c455f2e None admin] PATH_INFO: `/v3/services/placement` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:19.875296 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103cc7b5-bde9-43cf-ac21-bd0a4c455f2e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:19.876653 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103cc7b5-bde9-43cf-ac21-bd0a4c455f2e None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:19.889156 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application [None req-103cc7b5-bde9-43cf-ac21-bd0a4c455f2e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 08:31:19.889156 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:31:19.889156 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 08:31:19.889156 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 08:31:19.889156 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 08:31:19.889156 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 08:31:19.889156 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 08:31:19.889156 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:31:19.889156 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 08:31:19.889156 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:31:19.889156 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:31:19.889156 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:31:19.889156 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:31:19.889156 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:31:19.889156 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:31:19.889156 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:31:19.889156 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:31:19.890358 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:31:19.890358 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:31:19.890358 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:31:19.890358 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:31:19.890358 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:31:19.890358 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:31:19.890358 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 08:31:19.890358 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 08:31:19.890358 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 08:31:19.890358 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 08:31:19.890358 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:31:19.890358 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:31:19.890358 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:31:19.890358 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:31:19.890358 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:31:19.890358 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:31:19.891226 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:31:19.891226 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application with Lock( Jun 24 08:31:19.891226 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:31:19.891226 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:31:19.891226 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:31:19.891226 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:31:19.891226 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:31:19.891226 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:31:19.891226 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:31:19.891226 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:31:19.891226 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 08:31:19.891226 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:31:19.891226 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 08:31:19.891226 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: ERROR keystone.server.flask.application Jun 24 08:31:19.891226 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 663/1304] 10.222.0.13 () {56 vars in 1117 bytes} [Fri Jun 24 08:31:19 2022] GET /identity/v3/services/placement => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:31:19.896397 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b286dee-345a-4471-b1f8-d310522b3ea8 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:19.970146 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:31:19.992460 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b286dee-345a-4471-b1f8-d310522b3ea8 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:20.026414 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f76075-b3ef-4e6b-88cf-e4b915a2c942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:20.027071 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f76075-b3ef-4e6b-88cf-e4b915a2c942 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:20.027355 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f76075-b3ef-4e6b-88cf-e4b915a2c942 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:20.027630 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f76075-b3ef-4e6b-88cf-e4b915a2c942 None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:20.028333 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f76075-b3ef-4e6b-88cf-e4b915a2c942 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:31:20.028636 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f76075-b3ef-4e6b-88cf-e4b915a2c942 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:20.030033 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f76075-b3ef-4e6b-88cf-e4b915a2c942 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:20.035142 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 642/1305] 10.222.0.13 () {56 vars in 1116 bytes} [Fri Jun 24 08:31:19 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:31:20.040665 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103cc7b5-bde9-43cf-ac21-bd0a4c455f2e None admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:20.139040 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103cc7b5-bde9-43cf-ac21-bd0a4c455f2e None admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:20.170631 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceb285f-7307-4d41-b4b1-2e3b563cfcc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:20.171257 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb285f-7307-4d41-b4b1-2e3b563cfcc9 None admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:20.171537 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb285f-7307-4d41-b4b1-2e3b563cfcc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:20.171810 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb285f-7307-4d41-b4b1-2e3b563cfcc9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:20.172541 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceb285f-7307-4d41-b4b1-2e3b563cfcc9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:20.173913 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceb285f-7307-4d41-b4b1-2e3b563cfcc9 None admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:20.191793 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 664/1306] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Jun 24 08:31:20 2022] POST /identity/v3/endpoints => generated 331 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:31:30.032076 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=30 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:31:30.187996 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:24.349114 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e4af8-a9a8-439d-9686-05eec25fcc8a None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:24.356033 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e4af8-a9a8-439d-9686-05eec25fcc8a None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:24.356578 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e4af8-a9a8-439d-9686-05eec25fcc8a None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:24.362260 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 643/1307] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:32:24 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:32:24.364226 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d920fd-ab6a-4513-abe1-a5f0fcf74eca None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:24.364672 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d920fd-ab6a-4513-abe1-a5f0fcf74eca None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:24.364869 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d920fd-ab6a-4513-abe1-a5f0fcf74eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:24.469309 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-b8d920fd-ab6a-4513-abe1-a5f0fcf74eca None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:32:24.470833 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b8d920fd-ab6a-4513-abe1-a5f0fcf74eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:32:24.690888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 665/1308] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:32:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:32:24.748371 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c3a86e-4b95-4847-94b2-ca506c6ff557 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:24.748982 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c3a86e-4b95-4847-94b2-ca506c6ff557 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:24.749446 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c3a86e-4b95-4847-94b2-ca506c6ff557 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:24.751520 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 644/1309] 10.222.0.13 () {54 vars in 964 bytes} [Fri Jun 24 08:32:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:32:24.758525 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285783c9-93ef-4acc-b728-cdcaf09268a8 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:24.759259 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285783c9-93ef-4acc-b728-cdcaf09268a8 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:24.759691 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285783c9-93ef-4acc-b728-cdcaf09268a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:24.862531 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-285783c9-93ef-4acc-b728-cdcaf09268a8 None None] MFA Rules not processed for user `c5f7ba6c7faf43e7acbb43eee58d8c8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:32:24.863656 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-285783c9-93ef-4acc-b728-cdcaf09268a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:32:24.884888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:24.948307 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 666/1310] 10.222.0.13 () {58 vars in 1061 bytes} [Fri Jun 24 08:32:24 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:32:24.960810 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c3a86e-4b95-4847-94b2-ca506c6ff557 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:24.962882 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-f2c3a86e-4b95-4847-94b2-ca506c6ff557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:32:25.080124 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c3a86e-4b95-4847-94b2-ca506c6ff557 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:25.157925 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62dfb73-b17e-4af1-a53c-d1c3fd136eec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:25.158656 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62dfb73-b17e-4af1-a53c-d1c3fd136eec service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:25.159041 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62dfb73-b17e-4af1-a53c-d1c3fd136eec service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:25.159418 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62dfb73-b17e-4af1-a53c-d1c3fd136eec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:25.161167 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d62dfb73-b17e-4af1-a53c-d1c3fd136eec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:32:25.218878 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:25.279543 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62dfb73-b17e-4af1-a53c-d1c3fd136eec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:25.281184 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62dfb73-b17e-4af1-a53c-d1c3fd136eec service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:25.303679 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 645/1311] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:32:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:32:27.686721 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62309ddd-42b8-40cd-9561-d500d6e3f077 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:27.687281 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62309ddd-42b8-40cd-9561-d500d6e3f077 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:27.687281 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62309ddd-42b8-40cd-9561-d500d6e3f077 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:27.688817 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 667/1312] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:32:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:32:27.700546 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2bbbd-f5c0-4331-a074-13a34c12a008 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:27.701080 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2bbbd-f5c0-4331-a074-13a34c12a008 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:27.701080 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2bbbd-f5c0-4331-a074-13a34c12a008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:27.787117 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-51b2bbbd-f5c0-4331-a074-13a34c12a008 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:32:27.788363 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-51b2bbbd-f5c0-4331-a074-13a34c12a008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:32:27.904544 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 646/1313] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:32:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:32:27.936071 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62309ddd-42b8-40cd-9561-d500d6e3f077 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:27.937712 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-62309ddd-42b8-40cd-9561-d500d6e3f077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:32:28.056582 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62309ddd-42b8-40cd-9561-d500d6e3f077 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:28.141362 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:28.144672 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152cd898-8b80-4bb2-89d1-484d89fc073c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:28.145312 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152cd898-8b80-4bb2-89d1-484d89fc073c service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:28.145896 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152cd898-8b80-4bb2-89d1-484d89fc073c service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:28.146364 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152cd898-8b80-4bb2-89d1-484d89fc073c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:28.148250 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-152cd898-8b80-4bb2-89d1-484d89fc073c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:32:28.259107 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152cd898-8b80-4bb2-89d1-484d89fc073c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:28.260583 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152cd898-8b80-4bb2-89d1-484d89fc073c service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:28.280933 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 668/1314] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:32:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:32:30.655597 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bbe15e-3080-4e68-80ea-434a75f955bd None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:30.656126 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bbe15e-3080-4e68-80ea-434a75f955bd None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:30.656126 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bbe15e-3080-4e68-80ea-434a75f955bd None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:30.657459 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 647/1315] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:32:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:32:30.662797 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3902fa9-d705-4ef9-8f0b-53db7534623a None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:30.663230 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3902fa9-d705-4ef9-8f0b-53db7534623a None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:30.663362 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3902fa9-d705-4ef9-8f0b-53db7534623a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:30.737731 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-e3902fa9-d705-4ef9-8f0b-53db7534623a None None] MFA Rules not processed for user `8e322e2cfd9e4eca9f58d562b4da2dad`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:32:30.739401 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-e3902fa9-d705-4ef9-8f0b-53db7534623a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:32:30.791219 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:30.837077 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 669/1316] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:32:30 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:32:30.877780 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bbe15e-3080-4e68-80ea-434a75f955bd None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:30.879316 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-21bbe15e-3080-4e68-80ea-434a75f955bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:32:30.889938 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:30.987603 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bbe15e-3080-4e68-80ea-434a75f955bd None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:31.067962 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eb2433-258f-474c-97ec-fdebcfd92f01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:31.068608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eb2433-258f-474c-97ec-fdebcfd92f01 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:31.068896 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eb2433-258f-474c-97ec-fdebcfd92f01 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:31.069175 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eb2433-258f-474c-97ec-fdebcfd92f01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:31.070867 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-11eb2433-258f-474c-97ec-fdebcfd92f01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:32:31.137828 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:31.193740 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eb2433-258f-474c-97ec-fdebcfd92f01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:31.195186 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eb2433-258f-474c-97ec-fdebcfd92f01 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:31.213486 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 648/1317] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:32:30 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:32:33.855307 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f019ce-0ed1-4aa9-8132-b10e306b8b6a None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:33.855997 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f019ce-0ed1-4aa9-8132-b10e306b8b6a None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:33.855997 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f019ce-0ed1-4aa9-8132-b10e306b8b6a None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:33.857264 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 670/1318] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:32:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:32:33.862936 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515dac3c-770f-43cd-8140-ec931ed4ffef None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:33.863379 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515dac3c-770f-43cd-8140-ec931ed4ffef None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:33.863481 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515dac3c-770f-43cd-8140-ec931ed4ffef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:33.931770 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-515dac3c-770f-43cd-8140-ec931ed4ffef None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:32:33.933019 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-515dac3c-770f-43cd-8140-ec931ed4ffef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:32:34.038011 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 649/1319] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:32:33 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:32:34.093676 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f78ee8f-896c-400d-8d5e-2bc5a76c1e8b None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:34.093995 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f78ee8f-896c-400d-8d5e-2bc5a76c1e8b None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:34.094272 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f78ee8f-896c-400d-8d5e-2bc5a76c1e8b None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:34.095639 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 671/1320] 10.222.0.13 () {54 vars in 964 bytes} [Fri Jun 24 08:32:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:32:34.101934 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af3184-b619-4297-b221-3c1adfb0291a None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:34.102113 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af3184-b619-4297-b221-3c1adfb0291a None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:34.102389 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af3184-b619-4297-b221-3c1adfb0291a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:34.124061 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:34.176880 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-15af3184-b619-4297-b221-3c1adfb0291a None None] MFA Rules not processed for user `c5f7ba6c7faf43e7acbb43eee58d8c8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:32:34.178102 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-15af3184-b619-4297-b221-3c1adfb0291a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:32:34.252086 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 650/1321] 10.222.0.13 () {58 vars in 1061 bytes} [Fri Jun 24 08:32:34 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:32:34.262423 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f78ee8f-896c-400d-8d5e-2bc5a76c1e8b None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:34.263948 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-5f78ee8f-896c-400d-8d5e-2bc5a76c1e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:32:34.383621 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f78ee8f-896c-400d-8d5e-2bc5a76c1e8b None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:34.455986 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bf1a25-765c-4cfc-b1ea-03d3f2e7415d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:34.456625 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf1a25-765c-4cfc-b1ea-03d3f2e7415d service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:34.456904 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf1a25-765c-4cfc-b1ea-03d3f2e7415d service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:34.457209 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf1a25-765c-4cfc-b1ea-03d3f2e7415d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:34.458858 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-c2bf1a25-765c-4cfc-b1ea-03d3f2e7415d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:32:34.467201 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:34.566857 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bf1a25-765c-4cfc-b1ea-03d3f2e7415d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:34.568277 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bf1a25-765c-4cfc-b1ea-03d3f2e7415d service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:34.586075 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 672/1322] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:32:34 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:32:37.767231 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e93c5-3de5-4411-b585-3d062acef4d3 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:37.767843 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e93c5-3de5-4411-b585-3d062acef4d3 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:37.767843 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e93c5-3de5-4411-b585-3d062acef4d3 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:37.769214 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 651/1323] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:32:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:32:37.775247 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75844da6-bb7a-4caf-9504-19078973b992 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:37.775961 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75844da6-bb7a-4caf-9504-19078973b992 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:37.776304 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75844da6-bb7a-4caf-9504-19078973b992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:37.844412 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-75844da6-bb7a-4caf-9504-19078973b992 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:32:37.845651 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-75844da6-bb7a-4caf-9504-19078973b992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:32:37.908154 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:37.950981 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 673/1324] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:32:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:32:37.988969 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23e93c5-3de5-4411-b585-3d062acef4d3 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:37.990888 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d23e93c5-3de5-4411-b585-3d062acef4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:32:38.116325 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23e93c5-3de5-4411-b585-3d062acef4d3 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:38.130769 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:38.199629 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7cc847-5e17-4128-b6f3-d44b7e00e1c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:38.200285 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7cc847-5e17-4128-b6f3-d44b7e00e1c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:38.200571 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7cc847-5e17-4128-b6f3-d44b7e00e1c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:38.200843 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7cc847-5e17-4128-b6f3-d44b7e00e1c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:38.202518 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-fb7cc847-5e17-4128-b6f3-d44b7e00e1c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:32:38.315566 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7cc847-5e17-4128-b6f3-d44b7e00e1c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:38.317067 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7cc847-5e17-4128-b6f3-d44b7e00e1c3 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:38.336051 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 652/1325] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:32:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:32:40.474508 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298a41d-57ea-461a-a545-acbeb3392018 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:40.475053 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298a41d-57ea-461a-a545-acbeb3392018 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:40.475053 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298a41d-57ea-461a-a545-acbeb3392018 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:40.476380 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 674/1326] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:32:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:32:40.482489 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0987da-758e-445b-9ceb-555be3a0ed0e None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:40.482910 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0987da-758e-445b-9ceb-555be3a0ed0e None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:40.483058 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0987da-758e-445b-9ceb-555be3a0ed0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:40.530777 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:40.549564 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-5c0987da-758e-445b-9ceb-555be3a0ed0e None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:32:40.550819 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-5c0987da-758e-445b-9ceb-555be3a0ed0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:32:40.654157 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 653/1327] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:32:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:32:40.690428 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e298a41d-57ea-461a-a545-acbeb3392018 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:40.692046 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-e298a41d-57ea-461a-a545-acbeb3392018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:32:40.796232 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e298a41d-57ea-461a-a545-acbeb3392018 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:40.870166 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb9ec59-f92c-4f0b-8a01-03aef366bcda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:40.871121 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb9ec59-f92c-4f0b-8a01-03aef366bcda service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:40.871427 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb9ec59-f92c-4f0b-8a01-03aef366bcda service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:40.871710 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb9ec59-f92c-4f0b-8a01-03aef366bcda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:40.873483 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-beb9ec59-f92c-4f0b-8a01-03aef366bcda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:32:40.893649 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:40.994726 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb9ec59-f92c-4f0b-8a01-03aef366bcda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:40.997241 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb9ec59-f92c-4f0b-8a01-03aef366bcda service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:41.016432 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 675/1328] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:32:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:32:43.129618 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32083b6a-3a7d-444a-b72c-be3393ce1c11 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:43.130175 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32083b6a-3a7d-444a-b72c-be3393ce1c11 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:43.130175 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32083b6a-3a7d-444a-b72c-be3393ce1c11 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:43.131573 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 654/1329] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:32:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:32:43.137205 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3b618c-c322-4488-8325-6c1c2f20d400 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:43.137978 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3b618c-c322-4488-8325-6c1c2f20d400 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:43.138079 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3b618c-c322-4488-8325-6c1c2f20d400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:43.204157 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-fb3b618c-c322-4488-8325-6c1c2f20d400 None None] MFA Rules not processed for user `8e322e2cfd9e4eca9f58d562b4da2dad`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:32:43.205356 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-fb3b618c-c322-4488-8325-6c1c2f20d400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:32:43.273334 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:43.308194 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 676/1330] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:32:43 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:32:43.346296 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32083b6a-3a7d-444a-b72c-be3393ce1c11 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:43.347917 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-32083b6a-3a7d-444a-b72c-be3393ce1c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:32:43.454295 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32083b6a-3a7d-444a-b72c-be3393ce1c11 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:43.475595 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:43.533021 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf1b2e1-f5f0-438a-b424-b4f80b7f125e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:43.533583 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf1b2e1-f5f0-438a-b424-b4f80b7f125e service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:43.533897 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf1b2e1-f5f0-438a-b424-b4f80b7f125e service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:43.534177 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf1b2e1-f5f0-438a-b424-b4f80b7f125e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:43.535837 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-faf1b2e1-f5f0-438a-b424-b4f80b7f125e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:32:43.650032 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf1b2e1-f5f0-438a-b424-b4f80b7f125e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:43.651488 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf1b2e1-f5f0-438a-b424-b4f80b7f125e service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:43.672582 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 655/1331] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:32:43 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:32:46.001509 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d546f1-254d-413a-b7fd-06f03cee5a51 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:46.002454 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d546f1-254d-413a-b7fd-06f03cee5a51 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:46.002630 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d546f1-254d-413a-b7fd-06f03cee5a51 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:46.004106 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 677/1332] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:32:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:32:46.009146 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076e558-cf85-47a3-82ef-dc01143f9da8 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:46.009559 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076e558-cf85-47a3-82ef-dc01143f9da8 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:46.009708 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076e558-cf85-47a3-82ef-dc01143f9da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:46.076682 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:46.082587 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-b076e558-cf85-47a3-82ef-dc01143f9da8 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:32:46.084035 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-b076e558-cf85-47a3-82ef-dc01143f9da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:32:46.186426 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 656/1333] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:32:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:32:46.223077 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d546f1-254d-413a-b7fd-06f03cee5a51 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:46.224689 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-05d546f1-254d-413a-b7fd-06f03cee5a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:32:46.332485 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d546f1-254d-413a-b7fd-06f03cee5a51 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:46.413716 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15749401-b996-449e-9e57-d9821472de5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:46.414383 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15749401-b996-449e-9e57-d9821472de5f service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:46.414858 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15749401-b996-449e-9e57-d9821472de5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:46.415132 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15749401-b996-449e-9e57-d9821472de5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:46.416786 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-15749401-b996-449e-9e57-d9821472de5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:32:46.445093 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:46.530584 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15749401-b996-449e-9e57-d9821472de5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:46.532071 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15749401-b996-449e-9e57-d9821472de5f service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:46.550416 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 678/1334] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:32:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:32:48.614660 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c4b37-7cff-4447-b58d-943a1a69adae None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:48.615217 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c4b37-7cff-4447-b58d-943a1a69adae None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:48.615217 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c4b37-7cff-4447-b58d-943a1a69adae None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:48.616585 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 657/1335] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:32:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:32:48.622344 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadd7e9f-b025-43d5-b7cd-5776c0f2d581 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:48.622770 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadd7e9f-b025-43d5-b7cd-5776c0f2d581 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:48.622908 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadd7e9f-b025-43d5-b7cd-5776c0f2d581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:48.690172 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-dadd7e9f-b025-43d5-b7cd-5776c0f2d581 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:32:48.691366 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-dadd7e9f-b025-43d5-b7cd-5776c0f2d581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:32:48.766879 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:48.790698 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 679/1336] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:32:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:32:48.831000 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19c4b37-7cff-4447-b58d-943a1a69adae None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:48.832558 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d19c4b37-7cff-4447-b58d-943a1a69adae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:32:48.940592 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19c4b37-7cff-4447-b58d-943a1a69adae None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:48.970454 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:49.015836 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d560ae-6f34-4aa0-8467-2478549fa2c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:49.016502 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d560ae-6f34-4aa0-8467-2478549fa2c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:49.016788 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d560ae-6f34-4aa0-8467-2478549fa2c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:49.017065 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d560ae-6f34-4aa0-8467-2478549fa2c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:49.018754 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d9d560ae-6f34-4aa0-8467-2478549fa2c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:32:49.126193 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d560ae-6f34-4aa0-8467-2478549fa2c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:49.127998 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d560ae-6f34-4aa0-8467-2478549fa2c0 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:49.150369 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 658/1337] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:32:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:32:51.566181 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18cf0ed-3cca-453c-928f-414ab6e9f89b None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:51.566686 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18cf0ed-3cca-453c-928f-414ab6e9f89b None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:51.566686 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18cf0ed-3cca-453c-928f-414ab6e9f89b None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:51.568024 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 680/1338] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:32:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:32:51.573189 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780ca52-d5ea-4395-9152-092fd51214ea None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:51.573606 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780ca52-d5ea-4395-9152-092fd51214ea None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:51.573833 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780ca52-d5ea-4395-9152-092fd51214ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:51.638535 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-8780ca52-d5ea-4395-9152-092fd51214ea None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:32:51.639753 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-8780ca52-d5ea-4395-9152-092fd51214ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:32:51.648129 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:51.736928 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 659/1339] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:32:51 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:32:51.773008 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18cf0ed-3cca-453c-928f-414ab6e9f89b None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:51.774568 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-e18cf0ed-3cca-453c-928f-414ab6e9f89b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:32:51.879796 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18cf0ed-3cca-453c-928f-414ab6e9f89b None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:51.955193 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036a9b74-63fb-4bb1-907a-85ffa3c97924 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:51.955970 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a9b74-63fb-4bb1-907a-85ffa3c97924 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:51.956284 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a9b74-63fb-4bb1-907a-85ffa3c97924 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:51.956642 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a9b74-63fb-4bb1-907a-85ffa3c97924 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:51.958500 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-036a9b74-63fb-4bb1-907a-85ffa3c97924 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:32:51.994886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:52.069951 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036a9b74-63fb-4bb1-907a-85ffa3c97924 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:52.071392 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036a9b74-63fb-4bb1-907a-85ffa3c97924 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:52.091599 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 681/1340] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:32:51 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:32:56.351179 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a2063-8b12-48a2-80d3-c3c220149ba0 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:56.351719 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a2063-8b12-48a2-80d3-c3c220149ba0 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:56.351719 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a2063-8b12-48a2-80d3-c3c220149ba0 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:56.353330 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 660/1341] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:32:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:32:56.358681 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93602be1-9c90-495b-8cc9-60ae840aac75 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:56.359158 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93602be1-9c90-495b-8cc9-60ae840aac75 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:56.359260 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93602be1-9c90-495b-8cc9-60ae840aac75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:56.435211 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-93602be1-9c90-495b-8cc9-60ae840aac75 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:32:56.437028 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-93602be1-9c90-495b-8cc9-60ae840aac75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:32:56.543020 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:56.559648 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 682/1342] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:32:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:32:56.598504 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67a2063-8b12-48a2-80d3-c3c220149ba0 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:56.600639 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-e67a2063-8b12-48a2-80d3-c3c220149ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:32:56.697789 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67a2063-8b12-48a2-80d3-c3c220149ba0 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:56.701343 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed085bb8-cd06-488c-b29f-5460cdd69156 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:56.702052 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed085bb8-cd06-488c-b29f-5460cdd69156 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:56.702341 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed085bb8-cd06-488c-b29f-5460cdd69156 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:56.702620 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed085bb8-cd06-488c-b29f-5460cdd69156 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:56.711733 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed085bb8-cd06-488c-b29f-5460cdd69156 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:56.713166 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed085bb8-cd06-488c-b29f-5460cdd69156 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:56.723544 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 661/1343] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:32:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:32:59.377265 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68962a-ad85-49fe-9536-fa9ff62de060 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:59.377767 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68962a-ad85-49fe-9536-fa9ff62de060 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:59.377935 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68962a-ad85-49fe-9536-fa9ff62de060 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:59.379305 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 683/1344] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:32:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:32:59.384672 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38900093-372f-47e7-9b58-511228b495b8 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:59.385138 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38900093-372f-47e7-9b58-511228b495b8 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:59.385276 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38900093-372f-47e7-9b58-511228b495b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:59.415403 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-38900093-372f-47e7-9b58-511228b495b8 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:32:59.416614 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-38900093-372f-47e7-9b58-511228b495b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:32:59.444530 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:59.496509 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 662/1345] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:32:59 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:32:59.536520 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f68962a-ad85-49fe-9536-fa9ff62de060 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:59.538306 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-0f68962a-ad85-49fe-9536-fa9ff62de060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:32:59.552020 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f68962a-ad85-49fe-9536-fa9ff62de060 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:59.555605 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d2e6d5-304c-4aae-9e5d-a46bc1b0467d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:59.556269 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d2e6d5-304c-4aae-9e5d-a46bc1b0467d service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:59.556553 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d2e6d5-304c-4aae-9e5d-a46bc1b0467d service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:59.556831 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d2e6d5-304c-4aae-9e5d-a46bc1b0467d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:59.569654 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d2e6d5-304c-4aae-9e5d-a46bc1b0467d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:59.571254 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d2e6d5-304c-4aae-9e5d-a46bc1b0467d service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:59.582816 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 684/1346] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:32:59 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:33:03.991445 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7cd37-c3bc-42ab-a9c0-db8afccee85c None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:03.992006 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7cd37-c3bc-42ab-a9c0-db8afccee85c None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:03.992006 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7cd37-c3bc-42ab-a9c0-db8afccee85c None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:03.993302 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 663/1347] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:33:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:33:03.999178 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d933a169-4f09-4041-937b-94f13ca2241a None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:03.999613 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d933a169-4f09-4041-937b-94f13ca2241a None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:03.999808 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d933a169-4f09-4041-937b-94f13ca2241a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:04.017971 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-d933a169-4f09-4041-937b-94f13ca2241a None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:33:04.019276 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-d933a169-4f09-4041-937b-94f13ca2241a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:04.027224 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 685/1348] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:33:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:33:04.065825 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c7cd37-c3bc-42ab-a9c0-db8afccee85c None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:33:04.076929 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c7cd37-c3bc-42ab-a9c0-db8afccee85c None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:33:04.080596 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf463e01-beb1-4de3-a324-ff0208c9fa99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:33:04.081378 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf463e01-beb1-4de3-a324-ff0208c9fa99 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:04.081663 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf463e01-beb1-4de3-a324-ff0208c9fa99 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:04.081966 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf463e01-beb1-4de3-a324-ff0208c9fa99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:04.091649 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf463e01-beb1-4de3-a324-ff0208c9fa99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:33:04.093154 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf463e01-beb1-4de3-a324-ff0208c9fa99 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:33:04.102826 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 664/1349] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:33:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:33:09.110509 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca6c30-b2e4-40be-9b06-d34e5f79bf60 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:09.110953 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca6c30-b2e4-40be-9b06-d34e5f79bf60 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:09.111050 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca6c30-b2e4-40be-9b06-d34e5f79bf60 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:09.112921 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 686/1350] 10.222.0.13 () {54 vars in 921 bytes} [Fri Jun 24 08:33:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:33:09.119893 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3790f0-5820-41f4-b2b0-1ae72606121b None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:09.120294 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3790f0-5820-41f4-b2b0-1ae72606121b None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:09.120568 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3790f0-5820-41f4-b2b0-1ae72606121b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:09.205337 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-1c3790f0-5820-41f4-b2b0-1ae72606121b None None] MFA Rules not processed for user `39cc268bad604dd08594e3b39c7ecf5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:33:09.206746 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-1c3790f0-5820-41f4-b2b0-1ae72606121b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:09.303332 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 665/1351] 10.222.0.13 () {58 vars in 1018 bytes} [Fri Jun 24 08:33:09 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:33:13.764080 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863049b3-117c-47f6-bdab-51391d84190d None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:13.764616 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863049b3-117c-47f6-bdab-51391d84190d None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:13.764616 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863049b3-117c-47f6-bdab-51391d84190d None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:13.766029 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 687/1352] 10.222.0.13 () {54 vars in 921 bytes} [Fri Jun 24 08:33:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:33:13.771468 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64fb9d-5925-4053-9431-58b6a38d4ba6 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:13.771922 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64fb9d-5925-4053-9431-58b6a38d4ba6 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:13.772022 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64fb9d-5925-4053-9431-58b6a38d4ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:13.791266 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-7d64fb9d-5925-4053-9431-58b6a38d4ba6 None None] MFA Rules not processed for user `39cc268bad604dd08594e3b39c7ecf5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:33:13.793007 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-7d64fb9d-5925-4053-9431-58b6a38d4ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:13.802912 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 666/1353] 10.222.0.13 () {58 vars in 1018 bytes} [Fri Jun 24 08:33:13 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:33:14.012498 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=13 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:33:15.501057 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a204cec-e249-4c8d-9440-3db8f16194fd None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:15.501754 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a204cec-e249-4c8d-9440-3db8f16194fd None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:15.501754 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a204cec-e249-4c8d-9440-3db8f16194fd None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:15.503300 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 688/1354] 10.222.0.13 () {54 vars in 921 bytes} [Fri Jun 24 08:33:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:33:15.508370 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15422f3-5d27-4a65-a26c-4730f930f106 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:15.508784 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15422f3-5d27-4a65-a26c-4730f930f106 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:15.509033 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15422f3-5d27-4a65-a26c-4730f930f106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:15.528068 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-c15422f3-5d27-4a65-a26c-4730f930f106 None None] MFA Rules not processed for user `39cc268bad604dd08594e3b39c7ecf5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:33:15.529329 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-c15422f3-5d27-4a65-a26c-4730f930f106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:15.538076 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 667/1355] 10.222.0.13 () {58 vars in 1018 bytes} [Fri Jun 24 08:33:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:33:19.134986 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f9c90-4a93-4413-9ac5-0358952f3d8c None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:19.135498 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f9c90-4a93-4413-9ac5-0358952f3d8c None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:19.135498 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f9c90-4a93-4413-9ac5-0358952f3d8c None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:19.138460 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 689/1356] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:33:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:33:19.144088 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744b22b-e4ce-401a-8045-83dfafd0a169 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:19.144947 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744b22b-e4ce-401a-8045-83dfafd0a169 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:19.145433 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744b22b-e4ce-401a-8045-83dfafd0a169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:19.172230 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-1744b22b-e4ce-401a-8045-83dfafd0a169 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:33:19.173443 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-1744b22b-e4ce-401a-8045-83dfafd0a169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:19.182221 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 668/1357] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:33:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:33:19.240575 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221a524f-9270-4a40-93b1-a3e89cf10ffd None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:19.241514 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221a524f-9270-4a40-93b1-a3e89cf10ffd None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:19.242150 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221a524f-9270-4a40-93b1-a3e89cf10ffd None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:19.245815 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 690/1358] 10.222.0.13 () {54 vars in 961 bytes} [Fri Jun 24 08:33:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:33:19.255569 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcab8e0-6f75-4874-8b33-13ef5e1eb5ae None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:19.255569 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcab8e0-6f75-4874-8b33-13ef5e1eb5ae None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:19.255569 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcab8e0-6f75-4874-8b33-13ef5e1eb5ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:19.302006 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:33:19.311076 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-fdcab8e0-6f75-4874-8b33-13ef5e1eb5ae None None] MFA Rules not processed for user `74bb1bc6821b4500b26f160ccae5b7d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:33:19.312266 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-fdcab8e0-6f75-4874-8b33-13ef5e1eb5ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:19.420882 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 669/1359] 10.222.0.13 () {58 vars in 1058 bytes} [Fri Jun 24 08:33:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:33:19.429047 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221a524f-9270-4a40-93b1-a3e89cf10ffd None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:33:19.444996 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221a524f-9270-4a40-93b1-a3e89cf10ffd None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:33:19.453009 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bbf4bd-30b9-4684-8688-862079b04dfe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74bb1bc6821b4500b26f160ccae5b7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:33:19.454006 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bbf4bd-30b9-4684-8688-862079b04dfe service nova] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:19.454494 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bbf4bd-30b9-4684-8688-862079b04dfe service nova] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:19.454996 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bbf4bd-30b9-4684-8688-862079b04dfe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:19.465030 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bbf4bd-30b9-4684-8688-862079b04dfe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:33:19.467007 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bbf4bd-30b9-4684-8688-862079b04dfe service nova] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:33:19.478126 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 691/1360] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:33:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:33:21.763073 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7b6542-4a7e-4fb0-8768-7d093c277809 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:21.763827 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7b6542-4a7e-4fb0-8768-7d093c277809 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:21.763827 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7b6542-4a7e-4fb0-8768-7d093c277809 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:21.765357 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 670/1361] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:33:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:33:21.770846 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d0f9e9-dd8d-427f-aee0-5a183a6f9c07 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:21.771451 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d0f9e9-dd8d-427f-aee0-5a183a6f9c07 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:21.771451 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d0f9e9-dd8d-427f-aee0-5a183a6f9c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:21.791614 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-39d0f9e9-dd8d-427f-aee0-5a183a6f9c07 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:33:21.793762 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-39d0f9e9-dd8d-427f-aee0-5a183a6f9c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:21.802830 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 692/1362] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:33:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:33:21.851360 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7b6542-4a7e-4fb0-8768-7d093c277809 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:33:21.865960 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7b6542-4a7e-4fb0-8768-7d093c277809 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:33:21.871820 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0eb181-36d3-4d91-afe5-821c5f6bc289 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74bb1bc6821b4500b26f160ccae5b7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:33:21.872786 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0eb181-36d3-4d91-afe5-821c5f6bc289 service nova] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:21.873206 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0eb181-36d3-4d91-afe5-821c5f6bc289 service nova] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:21.873612 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0eb181-36d3-4d91-afe5-821c5f6bc289 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:21.883514 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0eb181-36d3-4d91-afe5-821c5f6bc289 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:33:21.885264 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0eb181-36d3-4d91-afe5-821c5f6bc289 service nova] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:33:21.898454 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 671/1363] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:33:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:33:24.494369 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e5a329-48f7-47f6-893c-907e43993146 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:24.494885 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e5a329-48f7-47f6-893c-907e43993146 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:24.494980 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e5a329-48f7-47f6-893c-907e43993146 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:24.496416 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 693/1364] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:33:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:33:24.501782 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c6b59a-1dfb-4a64-9e07-b9c91565c089 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:24.502192 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c6b59a-1dfb-4a64-9e07-b9c91565c089 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:24.502454 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c6b59a-1dfb-4a64-9e07-b9c91565c089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:24.520352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-85c6b59a-1dfb-4a64-9e07-b9c91565c089 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:33:24.521515 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-85c6b59a-1dfb-4a64-9e07-b9c91565c089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:24.530030 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 672/1365] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:33:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:33:24.575334 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70d3e3-5b78-4050-bcd1-101d7d95b91d None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:24.575626 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70d3e3-5b78-4050-bcd1-101d7d95b91d None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:24.575907 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70d3e3-5b78-4050-bcd1-101d7d95b91d None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:24.577265 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 694/1366] 10.222.0.13 () {54 vars in 961 bytes} [Fri Jun 24 08:33:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:33:24.583197 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdbb4e7-54e9-4b45-a4bf-6275ff679a52 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:24.583744 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdbb4e7-54e9-4b45-a4bf-6275ff679a52 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:24.584232 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdbb4e7-54e9-4b45-a4bf-6275ff679a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:24.604498 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-6cdbb4e7-54e9-4b45-a4bf-6275ff679a52 None None] MFA Rules not processed for user `74bb1bc6821b4500b26f160ccae5b7d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:33:24.606784 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-6cdbb4e7-54e9-4b45-a4bf-6275ff679a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:24.615987 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 673/1367] 10.222.0.13 () {58 vars in 1058 bytes} [Fri Jun 24 08:33:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:33:24.624712 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d70d3e3-5b78-4050-bcd1-101d7d95b91d None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:33:24.638864 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d70d3e3-5b78-4050-bcd1-101d7d95b91d None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:33:24.644755 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d979fa5-5a4e-4d59-8d37-8a71216ac2c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74bb1bc6821b4500b26f160ccae5b7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:33:24.645384 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d979fa5-5a4e-4d59-8d37-8a71216ac2c2 service nova] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:24.645667 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d979fa5-5a4e-4d59-8d37-8a71216ac2c2 service nova] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:24.645970 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d979fa5-5a4e-4d59-8d37-8a71216ac2c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:24.654967 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d979fa5-5a4e-4d59-8d37-8a71216ac2c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:33:24.656535 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d979fa5-5a4e-4d59-8d37-8a71216ac2c2 service nova] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:33:24.665897 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 695/1368] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:33:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:33:26.841583 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64411934-ddd4-4154-8e41-49c0f1e18405 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:26.842220 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64411934-ddd4-4154-8e41-49c0f1e18405 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:26.842220 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64411934-ddd4-4154-8e41-49c0f1e18405 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:26.843723 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 674/1369] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:33:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:33:26.849183 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5178fb-e4fb-463c-9166-b6641efe322e None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:26.849606 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5178fb-e4fb-463c-9166-b6641efe322e None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:26.849748 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5178fb-e4fb-463c-9166-b6641efe322e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:26.869977 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-0f5178fb-e4fb-463c-9166-b6641efe322e None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:33:26.871544 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-0f5178fb-e4fb-463c-9166-b6641efe322e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:26.881725 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 696/1370] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:33:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:33:26.924037 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64411934-ddd4-4154-8e41-49c0f1e18405 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:33:26.940713 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64411934-ddd4-4154-8e41-49c0f1e18405 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:33:26.947894 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8339f4-26cc-4845-b1cb-7233c9a1db84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74bb1bc6821b4500b26f160ccae5b7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:33:26.948539 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8339f4-26cc-4845-b1cb-7233c9a1db84 service nova] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:26.949229 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8339f4-26cc-4845-b1cb-7233c9a1db84 service nova] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:26.949714 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8339f4-26cc-4845-b1cb-7233c9a1db84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:26.958721 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8339f4-26cc-4845-b1cb-7233c9a1db84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:33:26.960323 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8339f4-26cc-4845-b1cb-7233c9a1db84 service nova] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:33:26.971453 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 675/1371] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:33:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:33:27.409213 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb6ac5-6d98-437d-91d5-42eff2825f38 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:27.409506 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb6ac5-6d98-437d-91d5-42eff2825f38 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:27.410000 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb6ac5-6d98-437d-91d5-42eff2825f38 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:27.411553 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 697/1372] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:33:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:33:27.416839 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a8b93-0210-4914-91e7-60a897b8e02b None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:27.417309 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a8b93-0210-4914-91e7-60a897b8e02b None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:27.418424 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a8b93-0210-4914-91e7-60a897b8e02b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:27.437490 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-582a8b93-0210-4914-91e7-60a897b8e02b None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:33:27.438992 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-582a8b93-0210-4914-91e7-60a897b8e02b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:27.453732 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 676/1373] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:33:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:33:27.799819 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c878d91-f4e8-4b9a-b8a5-545bbadf52e0 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:27.800108 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c878d91-f4e8-4b9a-b8a5-545bbadf52e0 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:27.800384 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c878d91-f4e8-4b9a-b8a5-545bbadf52e0 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:27.802221 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 698/1374] 10.222.0.13 () {54 vars in 963 bytes} [Fri Jun 24 08:33:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:33:27.807655 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6da7cd-7372-4fe6-a7d8-19592922beef None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:27.807944 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6da7cd-7372-4fe6-a7d8-19592922beef None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:27.808846 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6da7cd-7372-4fe6-a7d8-19592922beef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:27.863679 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-3f6da7cd-7372-4fe6-a7d8-19592922beef None None] MFA Rules not processed for user `44dcdfb5ca1541f9b4967a1f8ddbb34b`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:33:27.865200 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-3f6da7cd-7372-4fe6-a7d8-19592922beef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:27.945837 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 677/1375] 10.222.0.13 () {58 vars in 1060 bytes} [Fri Jun 24 08:33:27 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:33:27.951799 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c878d91-f4e8-4b9a-b8a5-545bbadf52e0 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:33:27.962961 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c878d91-f4e8-4b9a-b8a5-545bbadf52e0 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:33:27.967050 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194e08bf-9a5a-408b-a826-a78c2e79c611 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:33:27.967692 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e08bf-9a5a-408b-a826-a78c2e79c611 service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:27.967977 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e08bf-9a5a-408b-a826-a78c2e79c611 service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:27.968249 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e08bf-9a5a-408b-a826-a78c2e79c611 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:27.977901 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194e08bf-9a5a-408b-a826-a78c2e79c611 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:33:27.979332 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194e08bf-9a5a-408b-a826-a78c2e79c611 service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:33:27.990281 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 699/1376] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:33:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:33:29.081507 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc875c-2c74-4f8f-a96d-8131dd3b86e1 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:29.082050 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc875c-2c74-4f8f-a96d-8131dd3b86e1 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:29.082050 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc875c-2c74-4f8f-a96d-8131dd3b86e1 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:29.083437 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 678/1377] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:33:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:33:29.088684 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d23bfd-721c-4571-905e-6ca203c4e4ea None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:29.089108 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d23bfd-721c-4571-905e-6ca203c4e4ea None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:29.089244 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d23bfd-721c-4571-905e-6ca203c4e4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:29.107759 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-a1d23bfd-721c-4571-905e-6ca203c4e4ea None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:33:29.108965 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-a1d23bfd-721c-4571-905e-6ca203c4e4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:29.116812 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 700/1378] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:33:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:33:29.161082 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fc875c-2c74-4f8f-a96d-8131dd3b86e1 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:33:29.176223 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fc875c-2c74-4f8f-a96d-8131dd3b86e1 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:33:29.181580 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ea64b7-d8f3-447d-8d6b-435dacec61b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74bb1bc6821b4500b26f160ccae5b7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:33:29.182281 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ea64b7-d8f3-447d-8d6b-435dacec61b2 service nova] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:29.182533 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ea64b7-d8f3-447d-8d6b-435dacec61b2 service nova] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:29.182813 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ea64b7-d8f3-447d-8d6b-435dacec61b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:29.191149 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ea64b7-d8f3-447d-8d6b-435dacec61b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:33:29.192745 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ea64b7-d8f3-447d-8d6b-435dacec61b2 service nova] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:33:29.196725 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:33:29.203332 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 679/1379] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:33:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:33:31.375719 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c764ac-6f64-4b7f-9418-abba669fdd95 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:31.376177 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c764ac-6f64-4b7f-9418-abba669fdd95 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:31.376276 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c764ac-6f64-4b7f-9418-abba669fdd95 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:31.380955 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 701/1380] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:33:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:33:31.384473 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eec630-09a7-4853-8acb-b73881520fbb None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:31.384843 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eec630-09a7-4853-8acb-b73881520fbb None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:31.384935 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eec630-09a7-4853-8acb-b73881520fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:31.405381 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-30eec630-09a7-4853-8acb-b73881520fbb None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:33:31.406599 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-30eec630-09a7-4853-8acb-b73881520fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:31.414523 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 680/1381] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:33:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:33:31.471262 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c764ac-6f64-4b7f-9418-abba669fdd95 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:33:31.488907 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c764ac-6f64-4b7f-9418-abba669fdd95 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:33:31.497016 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87054a01-aa85-4175-999c-5de6369a0de3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74bb1bc6821b4500b26f160ccae5b7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:33:31.498056 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87054a01-aa85-4175-999c-5de6369a0de3 service nova] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:31.498552 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87054a01-aa85-4175-999c-5de6369a0de3 service nova] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:31.499029 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87054a01-aa85-4175-999c-5de6369a0de3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:31.512966 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87054a01-aa85-4175-999c-5de6369a0de3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:33:31.515109 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87054a01-aa85-4175-999c-5de6369a0de3 service nova] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:33:31.529944 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 702/1382] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:33:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:33:33.646407 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd51dc9-bed7-4935-b862-07c01e754cdc None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:33.646924 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd51dc9-bed7-4935-b862-07c01e754cdc None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:33.646924 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd51dc9-bed7-4935-b862-07c01e754cdc None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:33.648247 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 681/1383] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:33:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:33:33.654243 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf7b6a-b365-44ff-b705-16fb75dbde65 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:33.654725 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf7b6a-b365-44ff-b705-16fb75dbde65 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:33.654826 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf7b6a-b365-44ff-b705-16fb75dbde65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:33.676362 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-14cf7b6a-b365-44ff-b705-16fb75dbde65 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:33:33.677574 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-14cf7b6a-b365-44ff-b705-16fb75dbde65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:33.687907 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 703/1384] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:33:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:33:33.737077 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd51dc9-bed7-4935-b862-07c01e754cdc None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:33:33.752833 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd51dc9-bed7-4935-b862-07c01e754cdc None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:33:33.758681 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44506f1e-ec2a-497f-83ef-2f6f183628ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74bb1bc6821b4500b26f160ccae5b7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:33:33.759446 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44506f1e-ec2a-497f-83ef-2f6f183628ba service nova] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:33.759834 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44506f1e-ec2a-497f-83ef-2f6f183628ba service nova] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:33.760222 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44506f1e-ec2a-497f-83ef-2f6f183628ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:33.770022 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44506f1e-ec2a-497f-83ef-2f6f183628ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:33:33.771722 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44506f1e-ec2a-497f-83ef-2f6f183628ba service nova] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:33:33.793894 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 682/1385] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:33:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:33:35.860869 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f2618-57e8-46de-ade7-67d1bbb0d65c None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:35.861450 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f2618-57e8-46de-ade7-67d1bbb0d65c None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:35.861450 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f2618-57e8-46de-ade7-67d1bbb0d65c None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:35.862859 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 704/1386] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:33:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:33:35.868100 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c229b-a862-4927-8bbd-8153601b89a5 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:35.868533 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c229b-a862-4927-8bbd-8153601b89a5 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:35.868694 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c229b-a862-4927-8bbd-8153601b89a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:35.888398 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-1f3c229b-a862-4927-8bbd-8153601b89a5 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:33:35.889866 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-1f3c229b-a862-4927-8bbd-8153601b89a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:35.897889 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 683/1387] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:33:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:33:35.942385 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255f2618-57e8-46de-ade7-67d1bbb0d65c None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:33:35.957763 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255f2618-57e8-46de-ade7-67d1bbb0d65c None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:33:35.963229 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8357cf-bfb4-4a0a-bb92-e65d8e993fe4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74bb1bc6821b4500b26f160ccae5b7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:33:35.963868 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8357cf-bfb4-4a0a-bb92-e65d8e993fe4 service nova] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:35.964153 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8357cf-bfb4-4a0a-bb92-e65d8e993fe4 service nova] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:35.964522 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8357cf-bfb4-4a0a-bb92-e65d8e993fe4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:35.972728 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8357cf-bfb4-4a0a-bb92-e65d8e993fe4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:33:35.974342 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8357cf-bfb4-4a0a-bb92-e65d8e993fe4 service nova] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:33:35.983865 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 705/1388] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:33:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:33:38.051412 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d400a7-2d5d-44d8-8e50-1107ef9554c4 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:38.051979 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d400a7-2d5d-44d8-8e50-1107ef9554c4 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:38.051979 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d400a7-2d5d-44d8-8e50-1107ef9554c4 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:38.053352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 684/1389] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:33:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:33:38.060181 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3297b3-8fdd-4844-9ddd-6dbcd6ec6129 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:38.060798 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3297b3-8fdd-4844-9ddd-6dbcd6ec6129 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:38.060798 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3297b3-8fdd-4844-9ddd-6dbcd6ec6129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:38.079431 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-7c3297b3-8fdd-4844-9ddd-6dbcd6ec6129 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:33:38.080639 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-7c3297b3-8fdd-4844-9ddd-6dbcd6ec6129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:38.088549 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 706/1390] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:33:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:33:38.133158 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d400a7-2d5d-44d8-8e50-1107ef9554c4 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:33:38.150465 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d400a7-2d5d-44d8-8e50-1107ef9554c4 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:33:38.156238 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2deb014-765d-45e6-b975-4a152252ce94 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74bb1bc6821b4500b26f160ccae5b7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:33:38.156921 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2deb014-765d-45e6-b975-4a152252ce94 service nova] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:38.157202 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2deb014-765d-45e6-b975-4a152252ce94 service nova] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:38.157474 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2deb014-765d-45e6-b975-4a152252ce94 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:38.166338 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2deb014-765d-45e6-b975-4a152252ce94 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:33:38.167966 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2deb014-765d-45e6-b975-4a152252ce94 service nova] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:33:38.177462 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 685/1391] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:33:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:33:40.241233 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b985e53-e3e6-4c03-9d3d-89cbc4abba89 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:40.241821 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b985e53-e3e6-4c03-9d3d-89cbc4abba89 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:40.241821 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b985e53-e3e6-4c03-9d3d-89cbc4abba89 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:40.243189 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 707/1392] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:33:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:33:40.249336 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f62d00-6eed-402d-a911-90867f78cfa7 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:40.249801 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f62d00-6eed-402d-a911-90867f78cfa7 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:40.249914 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f62d00-6eed-402d-a911-90867f78cfa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:40.268473 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-05f62d00-6eed-402d-a911-90867f78cfa7 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:33:40.269728 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-05f62d00-6eed-402d-a911-90867f78cfa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:40.278866 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 686/1393] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:33:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:33:40.325443 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b985e53-e3e6-4c03-9d3d-89cbc4abba89 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:33:40.340126 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b985e53-e3e6-4c03-9d3d-89cbc4abba89 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:33:40.346432 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef5758a-b2df-41fd-bc06-209a76dda942 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74bb1bc6821b4500b26f160ccae5b7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:33:40.347120 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef5758a-b2df-41fd-bc06-209a76dda942 service nova] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:40.347445 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef5758a-b2df-41fd-bc06-209a76dda942 service nova] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:40.347739 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef5758a-b2df-41fd-bc06-209a76dda942 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:40.357905 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef5758a-b2df-41fd-bc06-209a76dda942 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:33:40.359558 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef5758a-b2df-41fd-bc06-209a76dda942 service nova] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:33:40.371394 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 708/1394] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:33:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:33:43.100444 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec47b6-8051-49f2-a66b-d1b3e2bf9a20 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:43.101029 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec47b6-8051-49f2-a66b-d1b3e2bf9a20 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:43.101029 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec47b6-8051-49f2-a66b-d1b3e2bf9a20 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:43.102534 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 687/1395] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:33:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:33:43.108237 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bee694-9ece-4a52-8dae-2791fb7b2907 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:43.108710 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bee694-9ece-4a52-8dae-2791fb7b2907 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:43.108890 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bee694-9ece-4a52-8dae-2791fb7b2907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:43.128449 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-20bee694-9ece-4a52-8dae-2791fb7b2907 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:33:43.129786 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-20bee694-9ece-4a52-8dae-2791fb7b2907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:43.138895 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 709/1396] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:33:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:33:43.186685 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ec47b6-8051-49f2-a66b-d1b3e2bf9a20 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:33:43.202637 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ec47b6-8051-49f2-a66b-d1b3e2bf9a20 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:33:43.208210 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebaffe1-eed7-4c1e-b2a0-82382b90be7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74bb1bc6821b4500b26f160ccae5b7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:33:43.208870 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebaffe1-eed7-4c1e-b2a0-82382b90be7d service nova] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:43.209190 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebaffe1-eed7-4c1e-b2a0-82382b90be7d service nova] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:43.209501 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebaffe1-eed7-4c1e-b2a0-82382b90be7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:43.218541 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebaffe1-eed7-4c1e-b2a0-82382b90be7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:33:43.220196 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebaffe1-eed7-4c1e-b2a0-82382b90be7d service nova] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:33:43.230544 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 688/1397] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:33:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:33:45.378185 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c11c62-6c33-4fc5-9fa6-0fcaa50aa73f None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:45.378662 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c11c62-6c33-4fc5-9fa6-0fcaa50aa73f None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:45.378781 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c11c62-6c33-4fc5-9fa6-0fcaa50aa73f None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:45.380110 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 710/1398] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:33:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:33:45.385297 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293dcce9-36cf-4e88-ba6f-eb9e5e88fbd6 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:45.385752 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293dcce9-36cf-4e88-ba6f-eb9e5e88fbd6 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:45.385898 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293dcce9-36cf-4e88-ba6f-eb9e5e88fbd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:45.405907 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-293dcce9-36cf-4e88-ba6f-eb9e5e88fbd6 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:33:45.407111 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-293dcce9-36cf-4e88-ba6f-eb9e5e88fbd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:45.415097 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 689/1399] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:33:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:33:45.464145 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c11c62-6c33-4fc5-9fa6-0fcaa50aa73f None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:33:45.473735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=55 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:33:45.481507 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c11c62-6c33-4fc5-9fa6-0fcaa50aa73f None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:33:45.487084 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6684b401-73ed-437f-b4f7-d16c91976bf8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74bb1bc6821b4500b26f160ccae5b7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:33:45.487743 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6684b401-73ed-437f-b4f7-d16c91976bf8 service nova] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:45.488029 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6684b401-73ed-437f-b4f7-d16c91976bf8 service nova] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:45.488307 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6684b401-73ed-437f-b4f7-d16c91976bf8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:45.498893 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6684b401-73ed-437f-b4f7-d16c91976bf8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:33:45.500515 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6684b401-73ed-437f-b4f7-d16c91976bf8 service nova] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:33:45.511406 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 711/1400] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:33:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:33:45.864212 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa32eedc-a184-4176-b6e5-8f91ca4f16db None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:45.864497 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa32eedc-a184-4176-b6e5-8f91ca4f16db None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:45.864775 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa32eedc-a184-4176-b6e5-8f91ca4f16db None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:45.866241 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 690/1401] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:33:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:33:45.871928 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a48dc-304f-4dc6-b1d5-cb0c1b88831c None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:45.872216 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a48dc-304f-4dc6-b1d5-cb0c1b88831c None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:45.872493 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a48dc-304f-4dc6-b1d5-cb0c1b88831c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:45.889918 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-564a48dc-304f-4dc6-b1d5-cb0c1b88831c None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:33:45.891364 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-564a48dc-304f-4dc6-b1d5-cb0c1b88831c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:45.899220 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 712/1402] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:33:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:33:45.955649 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e9921-3b29-452a-ab8a-1d5454edc4a8 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:45.955934 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e9921-3b29-452a-ab8a-1d5454edc4a8 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:45.956208 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e9921-3b29-452a-ab8a-1d5454edc4a8 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:45.957601 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 691/1403] 10.222.0.13 () {54 vars in 962 bytes} [Fri Jun 24 08:33:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:33:45.962785 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f222c33b-2b75-41cf-a1fe-6843b684f307 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:45.963072 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f222c33b-2b75-41cf-a1fe-6843b684f307 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:45.963347 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f222c33b-2b75-41cf-a1fe-6843b684f307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:46.022622 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-f222c33b-2b75-41cf-a1fe-6843b684f307 None None] MFA Rules not processed for user `d15c5f6aeb874191b73c8ee0410c6bc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:33:46.023867 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-f222c33b-2b75-41cf-a1fe-6843b684f307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:46.094348 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 713/1404] 10.222.0.13 () {58 vars in 1059 bytes} [Fri Jun 24 08:33:45 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:33:46.104340 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42e9921-3b29-452a-ab8a-1d5454edc4a8 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:33:46.115868 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42e9921-3b29-452a-ab8a-1d5454edc4a8 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:33:46.119307 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f34fac-c406-4cd9-a90b-cc0ae63a752f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd15c5f6aeb874191b73c8ee0410c6bc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:33:46.119940 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f34fac-c406-4cd9-a90b-cc0ae63a752f service glance] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:46.120224 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f34fac-c406-4cd9-a90b-cc0ae63a752f service glance] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:46.120498 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f34fac-c406-4cd9-a90b-cc0ae63a752f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:46.130351 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f34fac-c406-4cd9-a90b-cc0ae63a752f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:33:46.131814 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f34fac-c406-4cd9-a90b-cc0ae63a752f service glance] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:33:46.141941 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 692/1405] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:33:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:33:46.849337 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427b303-3a68-4381-bbc0-d45f87b6f83d None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:46.849850 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427b303-3a68-4381-bbc0-d45f87b6f83d None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:46.849942 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427b303-3a68-4381-bbc0-d45f87b6f83d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:46.897981 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-b427b303-3a68-4381-bbc0-d45f87b6f83d None None] MFA Rules not processed for user `85d9c4c57681447c898ebfd7990b05b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:33:46.899100 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b427b303-3a68-4381-bbc0-d45f87b6f83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:46.965336 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 714/1406] 10.222.0.13 () {58 vars in 1009 bytes} [Fri Jun 24 08:33:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:33:47.006205 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585cccd-2d78-468a-abee-31f8d4f7cf5e None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:47.006694 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585cccd-2d78-468a-abee-31f8d4f7cf5e None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:47.006803 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585cccd-2d78-468a-abee-31f8d4f7cf5e None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:47.008204 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 693/1407] 10.222.0.13 () {54 vars in 942 bytes} [Fri Jun 24 08:33:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:33:47.015512 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b09de9-7aa2-478d-a0d2-e00dc564a50f None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:47.015803 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b09de9-7aa2-478d-a0d2-e00dc564a50f None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:47.016083 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b09de9-7aa2-478d-a0d2-e00dc564a50f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:47.067165 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:33:47.070292 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-32b09de9-7aa2-478d-a0d2-e00dc564a50f None None] MFA Rules not processed for user `bec8c2f113cf41a0939b81f38fbee813`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:33:47.071858 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-32b09de9-7aa2-478d-a0d2-e00dc564a50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:47.142571 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 715/1408] 10.222.0.13 () {58 vars in 1039 bytes} [Fri Jun 24 08:33:47 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:33:47.151679 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e585cccd-2d78-468a-abee-31f8d4f7cf5e None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:33:47.167062 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e585cccd-2d78-468a-abee-31f8d4f7cf5e None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:33:47.171904 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae95f435-d431-4801-a4f2-81221c0c03f6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec8c2f113cf41a0939b81f38fbee813', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:33:47.172536 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95f435-d431-4801-a4f2-81221c0c03f6 service swift] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:47.172820 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95f435-d431-4801-a4f2-81221c0c03f6 service swift] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:47.173095 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95f435-d431-4801-a4f2-81221c0c03f6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:47.180995 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae95f435-d431-4801-a4f2-81221c0c03f6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:33:47.182627 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae95f435-d431-4801-a4f2-81221c0c03f6 service swift] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:33:47.191168 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 694/1409] 10.222.0.13 () {60 vars in 1384 bytes} [Fri Jun 24 08:33:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 08:33:57.139390 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=30 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:33:57.187975 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=43 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:34:09.919087 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5496097-3773-4d7b-b470-a547b3886dbd None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:09.919676 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5496097-3773-4d7b-b470-a547b3886dbd None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:09.919676 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5496097-3773-4d7b-b470-a547b3886dbd None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:09.921008 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 716/1410] 10.222.0.13 () {54 vars in 918 bytes} [Fri Jun 24 08:34:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:34:09.926432 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b819b824-4de5-4859-a0b5-1780df849cc9 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:09.926943 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b819b824-4de5-4859-a0b5-1780df849cc9 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:09.927059 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b819b824-4de5-4859-a0b5-1780df849cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:09.946276 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-b819b824-4de5-4859-a0b5-1780df849cc9 None None] MFA Rules not processed for user `39cc268bad604dd08594e3b39c7ecf5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:34:09.947981 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-b819b824-4de5-4859-a0b5-1780df849cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:34:09.956571 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 695/1411] 10.222.0.13 () {58 vars in 1015 bytes} [Fri Jun 24 08:34:09 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:34:09.971681 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c220d29-05a2-4de5-91d0-0b53996ddf44 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:09.971961 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c220d29-05a2-4de5-91d0-0b53996ddf44 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:09.972238 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c220d29-05a2-4de5-91d0-0b53996ddf44 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:09.973589 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 717/1412] 10.222.0.13 () {54 vars in 918 bytes} [Fri Jun 24 08:34:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:34:09.978377 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82bc5e6-6526-4ee2-b04f-9237ba688217 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:09.978721 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82bc5e6-6526-4ee2-b04f-9237ba688217 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:09.979109 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82bc5e6-6526-4ee2-b04f-9237ba688217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:09.996795 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-c82bc5e6-6526-4ee2-b04f-9237ba688217 None None] MFA Rules not processed for user `74bb1bc6821b4500b26f160ccae5b7d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:34:09.998012 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-c82bc5e6-6526-4ee2-b04f-9237ba688217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:34:10.007063 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 696/1413] 10.222.0.13 () {58 vars in 1015 bytes} [Fri Jun 24 08:34:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:34:12.500042 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36806ff9-cc16-4019-ae48-e5b89042adde None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:12.500602 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36806ff9-cc16-4019-ae48-e5b89042adde None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:12.500602 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36806ff9-cc16-4019-ae48-e5b89042adde None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:12.501980 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 718/1414] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 08:34:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:34:12.507069 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e923c09-074e-48c3-b426-c40fa3e08753 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:12.507473 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e923c09-074e-48c3-b426-c40fa3e08753 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:12.507621 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e923c09-074e-48c3-b426-c40fa3e08753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:12.525760 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-6e923c09-074e-48c3-b426-c40fa3e08753 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:34:12.526945 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-6e923c09-074e-48c3-b426-c40fa3e08753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:34:12.534710 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 697/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 08:34:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:34:12.578148 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36806ff9-cc16-4019-ae48-e5b89042adde None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:34:12.593323 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36806ff9-cc16-4019-ae48-e5b89042adde None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:34:12.599396 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736961fd-05d3-4d8d-a8cd-d579fd77b564 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74bb1bc6821b4500b26f160ccae5b7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:34:12.600042 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736961fd-05d3-4d8d-a8cd-d579fd77b564 service nova] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:12.600326 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736961fd-05d3-4d8d-a8cd-d579fd77b564 service nova] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:12.600603 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736961fd-05d3-4d8d-a8cd-d579fd77b564 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:12.609140 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736961fd-05d3-4d8d-a8cd-d579fd77b564 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:34:12.610769 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736961fd-05d3-4d8d-a8cd-d579fd77b564 service nova] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:34:12.620862 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 719/1416] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:34:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:34:13.547765 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5261695e-031e-4a4e-b43d-2af937c00418 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:13.548303 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5261695e-031e-4a4e-b43d-2af937c00418 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:13.548425 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5261695e-031e-4a4e-b43d-2af937c00418 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:13.549774 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 698/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 08:34:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:34:13.555028 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b8e31-c88c-4440-b77e-58ac382af7bb None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:13.555475 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b8e31-c88c-4440-b77e-58ac382af7bb None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:13.555632 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b8e31-c88c-4440-b77e-58ac382af7bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:13.574576 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-760b8e31-c88c-4440-b77e-58ac382af7bb None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:34:13.575786 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-760b8e31-c88c-4440-b77e-58ac382af7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:34:13.583935 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 720/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 08:34:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:34:13.733654 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5261695e-031e-4a4e-b43d-2af937c00418 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:34:13.748877 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5261695e-031e-4a4e-b43d-2af937c00418 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:34:13.754304 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150d9ffe-0ce0-495d-8215-456d7497d1dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74bb1bc6821b4500b26f160ccae5b7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:34:13.754946 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d9ffe-0ce0-495d-8215-456d7497d1dd service nova] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:13.755232 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d9ffe-0ce0-495d-8215-456d7497d1dd service nova] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:13.755510 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d9ffe-0ce0-495d-8215-456d7497d1dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:13.765140 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150d9ffe-0ce0-495d-8215-456d7497d1dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:34:13.766760 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150d9ffe-0ce0-495d-8215-456d7497d1dd service nova] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:34:13.777328 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 699/1419] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:34:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:34:14.663983 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2053b-c67f-4683-8382-18fdbfa9f74c None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:14.664559 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2053b-c67f-4683-8382-18fdbfa9f74c None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:14.664559 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2053b-c67f-4683-8382-18fdbfa9f74c None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:14.666489 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 721/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 08:34:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:34:14.672223 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4b4c4-9aab-4a09-a9cf-b9fd2e3f199b None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:14.672720 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4b4c4-9aab-4a09-a9cf-b9fd2e3f199b None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:14.672824 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4b4c4-9aab-4a09-a9cf-b9fd2e3f199b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:14.692348 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-7bf4b4c4-9aab-4a09-a9cf-b9fd2e3f199b None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:34:14.693549 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-7bf4b4c4-9aab-4a09-a9cf-b9fd2e3f199b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:34:14.701730 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 700/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 08:34:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:34:14.851790 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b2053b-c67f-4683-8382-18fdbfa9f74c None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:34:14.866541 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b2053b-c67f-4683-8382-18fdbfa9f74c None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:34:14.873180 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5832ca5-b040-4dae-ab92-8cbe8218a583 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74bb1bc6821b4500b26f160ccae5b7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:34:14.873861 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5832ca5-b040-4dae-ab92-8cbe8218a583 service nova] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:14.874141 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5832ca5-b040-4dae-ab92-8cbe8218a583 service nova] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:14.874415 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5832ca5-b040-4dae-ab92-8cbe8218a583 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:14.882566 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5832ca5-b040-4dae-ab92-8cbe8218a583 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:34:14.884160 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5832ca5-b040-4dae-ab92-8cbe8218a583 service nova] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:34:14.894466 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 722/1422] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:34:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:34:15.756516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db27a3-7277-4772-91bd-e27a5eb5196a None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:15.757107 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db27a3-7277-4772-91bd-e27a5eb5196a None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:15.757107 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db27a3-7277-4772-91bd-e27a5eb5196a None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:15.758504 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 701/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 08:34:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:34:15.764207 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f670040-2926-470f-bca2-3cc0243d91e3 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:15.764652 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f670040-2926-470f-bca2-3cc0243d91e3 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:15.764767 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f670040-2926-470f-bca2-3cc0243d91e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:15.782407 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-6f670040-2926-470f-bca2-3cc0243d91e3 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:34:15.783595 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-6f670040-2926-470f-bca2-3cc0243d91e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:34:15.791818 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 723/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 08:34:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:34:15.947036 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db27a3-7277-4772-91bd-e27a5eb5196a None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:34:15.961482 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db27a3-7277-4772-91bd-e27a5eb5196a None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:34:15.968739 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aebc8a-1541-4727-9687-8ead849c134a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74bb1bc6821b4500b26f160ccae5b7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:34:15.969396 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aebc8a-1541-4727-9687-8ead849c134a service nova] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:15.969684 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aebc8a-1541-4727-9687-8ead849c134a service nova] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:15.969991 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aebc8a-1541-4727-9687-8ead849c134a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:15.979495 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aebc8a-1541-4727-9687-8ead849c134a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:34:15.981095 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aebc8a-1541-4727-9687-8ead849c134a service nova] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:34:15.990708 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 702/1425] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:34:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:34:16.863310 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa3854a-de53-4839-aeb2-2397cb2e5c99 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:16.863912 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa3854a-de53-4839-aeb2-2397cb2e5c99 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:16.863912 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa3854a-de53-4839-aeb2-2397cb2e5c99 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:16.865305 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 724/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 08:34:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:34:16.871426 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5b422-2078-4a9d-b281-424c83b2c411 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:16.871887 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5b422-2078-4a9d-b281-424c83b2c411 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:16.871997 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5b422-2078-4a9d-b281-424c83b2c411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:16.890435 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-cbf5b422-2078-4a9d-b281-424c83b2c411 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:34:16.891652 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-cbf5b422-2078-4a9d-b281-424c83b2c411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:34:16.899605 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 703/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 08:34:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:34:17.052607 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa3854a-de53-4839-aeb2-2397cb2e5c99 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:34:17.067651 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa3854a-de53-4839-aeb2-2397cb2e5c99 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:34:17.073432 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3517ab9d-38a4-487d-977e-d850617dd140 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74bb1bc6821b4500b26f160ccae5b7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:34:17.074122 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3517ab9d-38a4-487d-977e-d850617dd140 service nova] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:17.074421 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3517ab9d-38a4-487d-977e-d850617dd140 service nova] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:17.074679 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3517ab9d-38a4-487d-977e-d850617dd140 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:17.089400 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3517ab9d-38a4-487d-977e-d850617dd140 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:34:17.091063 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3517ab9d-38a4-487d-977e-d850617dd140 service nova] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:34:17.101162 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 725/1428] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:34:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:34:17.906184 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa61ae7b-c89c-4585-987a-ca9191574a75 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:17.906723 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa61ae7b-c89c-4585-987a-ca9191574a75 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:17.906723 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa61ae7b-c89c-4585-987a-ca9191574a75 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:17.908102 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 704/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 08:34:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:34:17.913041 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c432f0-ec49-4c86-8313-5369b9376ed0 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:17.913454 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c432f0-ec49-4c86-8313-5369b9376ed0 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:17.913605 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c432f0-ec49-4c86-8313-5369b9376ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:17.932527 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-f1c432f0-ec49-4c86-8313-5369b9376ed0 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:34:17.933764 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-f1c432f0-ec49-4c86-8313-5369b9376ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:34:17.942317 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 726/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 08:34:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:34:18.093653 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa61ae7b-c89c-4585-987a-ca9191574a75 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:34:18.108649 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa61ae7b-c89c-4585-987a-ca9191574a75 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:34:18.114021 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def62b80-33b1-43d2-8017-cc02672588d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74bb1bc6821b4500b26f160ccae5b7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:34:18.114669 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def62b80-33b1-43d2-8017-cc02672588d9 service nova] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:18.114957 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def62b80-33b1-43d2-8017-cc02672588d9 service nova] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:18.115234 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def62b80-33b1-43d2-8017-cc02672588d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:18.123844 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def62b80-33b1-43d2-8017-cc02672588d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:34:18.125467 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def62b80-33b1-43d2-8017-cc02672588d9 service nova] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:34:18.135941 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 705/1431] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:34:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:34:18.951051 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d2298a-610d-43ab-90b2-a7fa2bea4960 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:18.951710 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d2298a-610d-43ab-90b2-a7fa2bea4960 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:18.951710 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d2298a-610d-43ab-90b2-a7fa2bea4960 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:18.952969 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 727/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 08:34:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:34:18.958033 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb965a-1dc6-47bb-9859-6538df203075 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:18.958446 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb965a-1dc6-47bb-9859-6538df203075 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:18.958586 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb965a-1dc6-47bb-9859-6538df203075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:18.976151 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-9bcb965a-1dc6-47bb-9859-6538df203075 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:34:18.978132 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-9bcb965a-1dc6-47bb-9859-6538df203075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:34:18.986151 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 706/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 08:34:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:34:19.171774 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d2298a-610d-43ab-90b2-a7fa2bea4960 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:34:19.186949 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d2298a-610d-43ab-90b2-a7fa2bea4960 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:34:19.192317 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbbe9a0-1288-4952-b6f0-19a0e340fe1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74bb1bc6821b4500b26f160ccae5b7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:34:19.192962 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbbe9a0-1288-4952-b6f0-19a0e340fe1d service nova] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:19.193242 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbbe9a0-1288-4952-b6f0-19a0e340fe1d service nova] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:19.193514 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbbe9a0-1288-4952-b6f0-19a0e340fe1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:19.202191 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbbe9a0-1288-4952-b6f0-19a0e340fe1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:34:19.203697 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbbe9a0-1288-4952-b6f0-19a0e340fe1d service nova] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:34:19.213641 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 728/1434] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:34:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:34:20.029315 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec2dd58-c4d6-449c-96d4-f002d4eca5df None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:20.029907 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec2dd58-c4d6-449c-96d4-f002d4eca5df None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:20.029907 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec2dd58-c4d6-449c-96d4-f002d4eca5df None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:20.031225 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 707/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 08:34:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:34:20.036245 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d651f4f-3e37-4383-ada8-83427860b8a0 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:20.036671 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d651f4f-3e37-4383-ada8-83427860b8a0 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:20.036831 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d651f4f-3e37-4383-ada8-83427860b8a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:20.055280 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-1d651f4f-3e37-4383-ada8-83427860b8a0 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:34:20.056708 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-1d651f4f-3e37-4383-ada8-83427860b8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:34:20.064764 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 729/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 08:34:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:34:20.220616 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec2dd58-c4d6-449c-96d4-f002d4eca5df None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:34:20.235996 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec2dd58-c4d6-449c-96d4-f002d4eca5df None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:34:20.241327 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89229be-cdf9-4949-8c44-be04943ac7d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74bb1bc6821b4500b26f160ccae5b7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:34:20.242001 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89229be-cdf9-4949-8c44-be04943ac7d9 service nova] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:20.242286 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89229be-cdf9-4949-8c44-be04943ac7d9 service nova] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:20.242559 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89229be-cdf9-4949-8c44-be04943ac7d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:20.252478 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89229be-cdf9-4949-8c44-be04943ac7d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:34:20.254380 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89229be-cdf9-4949-8c44-be04943ac7d9 service nova] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:34:20.265376 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 708/1437] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:34:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:34:21.105533 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a2744-24e5-4cb1-b78b-1eb4acaec41e None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:21.106166 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a2744-24e5-4cb1-b78b-1eb4acaec41e None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:21.106166 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a2744-24e5-4cb1-b78b-1eb4acaec41e None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:21.107713 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 730/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 08:34:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:34:21.114503 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b131f-96ef-4509-bb07-644f58cd0de5 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:21.114989 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b131f-96ef-4509-bb07-644f58cd0de5 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:21.115098 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b131f-96ef-4509-bb07-644f58cd0de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:21.133779 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-369b131f-96ef-4509-bb07-644f58cd0de5 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:34:21.134976 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-369b131f-96ef-4509-bb07-644f58cd0de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:34:21.142980 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 709/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 08:34:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:34:21.295156 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311a2744-24e5-4cb1-b78b-1eb4acaec41e None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:34:21.310802 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311a2744-24e5-4cb1-b78b-1eb4acaec41e None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:34:21.316251 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4733ee-36fa-48d8-8fe9-657a39a81f85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74bb1bc6821b4500b26f160ccae5b7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:34:21.316906 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4733ee-36fa-48d8-8fe9-657a39a81f85 service nova] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:21.317186 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4733ee-36fa-48d8-8fe9-657a39a81f85 service nova] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:21.317462 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4733ee-36fa-48d8-8fe9-657a39a81f85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:21.326752 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4733ee-36fa-48d8-8fe9-657a39a81f85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:34:21.328360 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4733ee-36fa-48d8-8fe9-657a39a81f85 service nova] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:34:21.339068 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 731/1440] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:34:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:34:23.293762 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bc7971-fbfa-44bc-a80d-5c0e0bac83bf None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:23.294587 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bc7971-fbfa-44bc-a80d-5c0e0bac83bf None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:23.294775 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bc7971-fbfa-44bc-a80d-5c0e0bac83bf None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:23.296201 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 710/1441] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 08:34:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:34:23.301552 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d76f9b-2d5f-4382-b35a-034c8fe28f8b None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:23.302041 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d76f9b-2d5f-4382-b35a-034c8fe28f8b None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:23.302147 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d76f9b-2d5f-4382-b35a-034c8fe28f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:23.320045 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-c4d76f9b-2d5f-4382-b35a-034c8fe28f8b None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:34:23.321516 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-c4d76f9b-2d5f-4382-b35a-034c8fe28f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:34:23.329215 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 732/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 08:34:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:34:23.368377 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bc7971-fbfa-44bc-a80d-5c0e0bac83bf None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:34:23.379063 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bc7971-fbfa-44bc-a80d-5c0e0bac83bf None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:34:23.382576 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e235fe54-564b-4775-8202-c479d2807db2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:34:23.383617 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e235fe54-564b-4775-8202-c479d2807db2 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:23.383903 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e235fe54-564b-4775-8202-c479d2807db2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:23.384181 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e235fe54-564b-4775-8202-c479d2807db2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:23.393491 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e235fe54-564b-4775-8202-c479d2807db2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:34:23.394965 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e235fe54-564b-4775-8202-c479d2807db2 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:34:23.404399 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 711/1443] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:34:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:34:25.624130 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03348a76-9c3d-4442-8ad4-019e5efef93c None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:25.624674 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03348a76-9c3d-4442-8ad4-019e5efef93c None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:25.624674 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03348a76-9c3d-4442-8ad4-019e5efef93c None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:25.626038 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 733/1444] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 08:34:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:34:25.631458 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb356d-9673-4a7e-bbf8-6f1a0efeef02 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:25.631903 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb356d-9673-4a7e-bbf8-6f1a0efeef02 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:25.632022 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb356d-9673-4a7e-bbf8-6f1a0efeef02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:25.651097 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-fbdb356d-9673-4a7e-bbf8-6f1a0efeef02 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:34:25.652312 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-fbdb356d-9673-4a7e-bbf8-6f1a0efeef02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:34:25.660209 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 712/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 08:34:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:34:25.698955 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03348a76-9c3d-4442-8ad4-019e5efef93c None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:34:25.709987 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03348a76-9c3d-4442-8ad4-019e5efef93c None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:34:25.715174 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabe76c1-a7d7-4272-9992-4c9215062e1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:34:25.715868 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe76c1-a7d7-4272-9992-4c9215062e1a service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:25.716307 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe76c1-a7d7-4272-9992-4c9215062e1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:25.716584 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe76c1-a7d7-4272-9992-4c9215062e1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:25.727125 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabe76c1-a7d7-4272-9992-4c9215062e1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:34:25.728658 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabe76c1-a7d7-4272-9992-4c9215062e1a service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:34:25.738989 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 734/1446] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:34:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:34:28.207289 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc91bf7-f1f5-45af-9108-d69d10c4fce5 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:28.207820 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc91bf7-f1f5-45af-9108-d69d10c4fce5 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:28.207820 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc91bf7-f1f5-45af-9108-d69d10c4fce5 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:28.209291 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 713/1447] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 08:34:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:34:28.215471 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a377e9-a6bd-482a-ad0d-738adb7fdddc None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:28.215923 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a377e9-a6bd-482a-ad0d-738adb7fdddc None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:28.216032 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a377e9-a6bd-482a-ad0d-738adb7fdddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:28.235089 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-43a377e9-a6bd-482a-ad0d-738adb7fdddc None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:34:28.236270 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-43a377e9-a6bd-482a-ad0d-738adb7fdddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:34:28.244062 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 735/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 08:34:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:34:28.282948 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc91bf7-f1f5-45af-9108-d69d10c4fce5 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:34:28.296243 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc91bf7-f1f5-45af-9108-d69d10c4fce5 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:34:28.300430 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8be2370-4e6a-4d9c-a2ad-b6e7a8a7f916 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:34:28.301069 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be2370-4e6a-4d9c-a2ad-b6e7a8a7f916 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:28.301357 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be2370-4e6a-4d9c-a2ad-b6e7a8a7f916 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:28.301636 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be2370-4e6a-4d9c-a2ad-b6e7a8a7f916 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:28.310903 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8be2370-4e6a-4d9c-a2ad-b6e7a8a7f916 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:34:28.312341 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8be2370-4e6a-4d9c-a2ad-b6e7a8a7f916 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:34:28.321900 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 714/1449] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:34:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:34:30.518283 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec5bf9c-dcc3-49cf-b0d9-f8847a139a7d None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:30.518850 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec5bf9c-dcc3-49cf-b0d9-f8847a139a7d None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:30.518850 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec5bf9c-dcc3-49cf-b0d9-f8847a139a7d None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:30.520190 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 736/1450] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 08:34:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:34:30.526021 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac07a12-49bd-4f6c-929c-6bac182dc5b2 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:30.526444 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac07a12-49bd-4f6c-929c-6bac182dc5b2 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:30.526585 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac07a12-49bd-4f6c-929c-6bac182dc5b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:30.544182 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-9ac07a12-49bd-4f6c-929c-6bac182dc5b2 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:34:30.545376 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-9ac07a12-49bd-4f6c-929c-6bac182dc5b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:34:30.553148 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 715/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 08:34:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:34:30.592010 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec5bf9c-dcc3-49cf-b0d9-f8847a139a7d None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:34:30.602623 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec5bf9c-dcc3-49cf-b0d9-f8847a139a7d None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:34:30.606049 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e79f1-aa40-495b-8231-499cadc5ddb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:34:30.606680 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e79f1-aa40-495b-8231-499cadc5ddb6 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:30.606959 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e79f1-aa40-495b-8231-499cadc5ddb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:30.607229 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e79f1-aa40-495b-8231-499cadc5ddb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:30.616149 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64e79f1-aa40-495b-8231-499cadc5ddb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:34:30.618394 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64e79f1-aa40-495b-8231-499cadc5ddb6 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:34:30.628066 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 737/1452] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:34:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:34:32.831874 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b588f8a-5e29-478e-9a19-301d0ec3dce2 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:32.832420 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b588f8a-5e29-478e-9a19-301d0ec3dce2 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:32.832537 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b588f8a-5e29-478e-9a19-301d0ec3dce2 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:32.834118 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 716/1453] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 08:34:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:34:32.840429 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f055646d-ad73-4c7d-80e3-569a2c685fe9 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:32.840904 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f055646d-ad73-4c7d-80e3-569a2c685fe9 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:32.841013 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f055646d-ad73-4c7d-80e3-569a2c685fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:32.859383 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-f055646d-ad73-4c7d-80e3-569a2c685fe9 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:34:32.860585 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-f055646d-ad73-4c7d-80e3-569a2c685fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:34:32.868774 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 738/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 08:34:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:34:32.907398 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b588f8a-5e29-478e-9a19-301d0ec3dce2 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:34:32.919121 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b588f8a-5e29-478e-9a19-301d0ec3dce2 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:34:32.922677 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d7f2b3-838d-484f-a3ae-167eca88481a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:34:32.923321 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d7f2b3-838d-484f-a3ae-167eca88481a service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:32.923606 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d7f2b3-838d-484f-a3ae-167eca88481a service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:32.923887 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d7f2b3-838d-484f-a3ae-167eca88481a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:32.933017 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d7f2b3-838d-484f-a3ae-167eca88481a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:34:32.934491 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d7f2b3-838d-484f-a3ae-167eca88481a service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:34:32.944040 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 717/1455] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:34:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:34:35.089894 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef08f21-cd3e-41d0-9297-5a4fd347e860 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:35.090440 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef08f21-cd3e-41d0-9297-5a4fd347e860 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:35.090440 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef08f21-cd3e-41d0-9297-5a4fd347e860 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:35.091804 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 739/1456] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 08:34:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:34:35.097263 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68175008-e49d-4e08-872c-2ec4f19d51ea None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:35.097684 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68175008-e49d-4e08-872c-2ec4f19d51ea None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:35.097859 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68175008-e49d-4e08-872c-2ec4f19d51ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:35.115506 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-68175008-e49d-4e08-872c-2ec4f19d51ea None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:34:35.117465 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-68175008-e49d-4e08-872c-2ec4f19d51ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:34:35.125353 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 718/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 08:34:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:34:35.164317 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef08f21-cd3e-41d0-9297-5a4fd347e860 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:34:35.174894 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef08f21-cd3e-41d0-9297-5a4fd347e860 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:34:35.178889 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabc46ed-56b8-441a-a1ce-50a367b8074b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:34:35.179685 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabc46ed-56b8-441a-a1ce-50a367b8074b service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:35.179968 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabc46ed-56b8-441a-a1ce-50a367b8074b service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:35.180242 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabc46ed-56b8-441a-a1ce-50a367b8074b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:35.189371 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabc46ed-56b8-441a-a1ce-50a367b8074b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:34:35.190834 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabc46ed-56b8-441a-a1ce-50a367b8074b service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:34:35.200931 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 740/1458] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:34:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:34:37.362428 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c89cc-c833-4b1f-9c43-2c86c9a01cf3 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:37.362971 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c89cc-c833-4b1f-9c43-2c86c9a01cf3 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:37.363092 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c89cc-c833-4b1f-9c43-2c86c9a01cf3 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:37.364407 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 719/1459] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 08:34:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:34:37.370890 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd6a4d-3ce0-4143-9f9e-aec56ec5086b None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:37.371350 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd6a4d-3ce0-4143-9f9e-aec56ec5086b None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:37.371479 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd6a4d-3ce0-4143-9f9e-aec56ec5086b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:37.377446 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=70 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:34:37.391944 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-c3bd6a4d-3ce0-4143-9f9e-aec56ec5086b None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:34:37.393151 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-c3bd6a4d-3ce0-4143-9f9e-aec56ec5086b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:34:37.401410 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 741/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 08:34:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:34:37.439932 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c89cc-c833-4b1f-9c43-2c86c9a01cf3 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:34:37.446699 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=73 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:34:37.453237 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c89cc-c833-4b1f-9c43-2c86c9a01cf3 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:34:37.456786 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc56747-42f2-40c9-b31c-d6970abdf559 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:34:37.457426 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc56747-42f2-40c9-b31c-d6970abdf559 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:37.457793 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc56747-42f2-40c9-b31c-d6970abdf559 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:37.458078 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc56747-42f2-40c9-b31c-d6970abdf559 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:37.467945 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc56747-42f2-40c9-b31c-d6970abdf559 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:34:37.469384 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc56747-42f2-40c9-b31c-d6970abdf559 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:34:37.479750 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 720/1461] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:34:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:34:39.644006 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0d568-6023-421d-84b6-4452512d72ad None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:39.644528 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0d568-6023-421d-84b6-4452512d72ad None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:39.644528 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0d568-6023-421d-84b6-4452512d72ad None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:39.645907 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 742/1462] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 08:34:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:34:39.651298 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c5ffbc-7ee1-4cd0-94d8-df862fe6886f None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:39.651842 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c5ffbc-7ee1-4cd0-94d8-df862fe6886f None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:39.651952 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c5ffbc-7ee1-4cd0-94d8-df862fe6886f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:39.669405 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-20c5ffbc-7ee1-4cd0-94d8-df862fe6886f None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:34:39.670629 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-20c5ffbc-7ee1-4cd0-94d8-df862fe6886f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:34:39.678979 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 721/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 08:34:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:34:39.716788 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0d568-6023-421d-84b6-4452512d72ad None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:34:39.729074 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0d568-6023-421d-84b6-4452512d72ad None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:34:39.732554 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd56c087-6171-4a96-97fa-7717f3682166 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:34:39.733232 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd56c087-6171-4a96-97fa-7717f3682166 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:39.733514 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd56c087-6171-4a96-97fa-7717f3682166 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:39.733832 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd56c087-6171-4a96-97fa-7717f3682166 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:39.744471 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd56c087-6171-4a96-97fa-7717f3682166 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:34:39.745957 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd56c087-6171-4a96-97fa-7717f3682166 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:34:39.756607 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 743/1464] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:34:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:34:41.869122 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d363cc8-e25e-4d2b-8a17-b4fededef645 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:41.869731 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d363cc8-e25e-4d2b-8a17-b4fededef645 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:41.869731 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d363cc8-e25e-4d2b-8a17-b4fededef645 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:41.871102 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 722/1465] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 08:34:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:34:41.876524 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f72c6e0-c632-4589-a2d8-488acfe80156 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:41.876946 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f72c6e0-c632-4589-a2d8-488acfe80156 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:41.877078 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f72c6e0-c632-4589-a2d8-488acfe80156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:41.895101 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-2f72c6e0-c632-4589-a2d8-488acfe80156 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:34:41.896778 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-2f72c6e0-c632-4589-a2d8-488acfe80156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:34:41.904675 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 744/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 08:34:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:34:41.941092 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d363cc8-e25e-4d2b-8a17-b4fededef645 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:34:41.951776 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d363cc8-e25e-4d2b-8a17-b4fededef645 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:34:41.955251 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8e9185-970a-4804-9897-3dc811f35e40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:34:41.955891 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8e9185-970a-4804-9897-3dc811f35e40 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:41.956176 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8e9185-970a-4804-9897-3dc811f35e40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:41.956453 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8e9185-970a-4804-9897-3dc811f35e40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:41.966107 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8e9185-970a-4804-9897-3dc811f35e40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:34:41.967540 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8e9185-970a-4804-9897-3dc811f35e40 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:34:41.977642 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 723/1467] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:34:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:34:44.080248 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c0ea6a-8e8b-4749-a757-f2802605aed8 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:44.080793 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c0ea6a-8e8b-4749-a757-f2802605aed8 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:44.080793 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c0ea6a-8e8b-4749-a757-f2802605aed8 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:44.082220 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 745/1468] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 08:34:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:34:44.088408 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0fddcb-03c8-4f97-be20-c7c885cf938b None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:44.088868 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0fddcb-03c8-4f97-be20-c7c885cf938b None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:44.088980 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0fddcb-03c8-4f97-be20-c7c885cf938b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:44.107928 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-2f0fddcb-03c8-4f97-be20-c7c885cf938b None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:34:44.109152 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-2f0fddcb-03c8-4f97-be20-c7c885cf938b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:34:44.117159 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 724/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 08:34:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:34:44.155211 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c0ea6a-8e8b-4749-a757-f2802605aed8 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:34:44.166684 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c0ea6a-8e8b-4749-a757-f2802605aed8 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:34:44.177107 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34953056-999c-4095-904e-fe6bb1145ec0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:34:44.177863 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34953056-999c-4095-904e-fe6bb1145ec0 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:44.178150 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34953056-999c-4095-904e-fe6bb1145ec0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:44.178421 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34953056-999c-4095-904e-fe6bb1145ec0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:44.188017 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34953056-999c-4095-904e-fe6bb1145ec0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:34:44.189442 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34953056-999c-4095-904e-fe6bb1145ec0 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:34:44.199256 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 746/1470] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:34:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:34:54.102508 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=17 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:34:54.194950 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=17 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:35:04.318714 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910ed2f-2fe3-4180-9476-bd6f97142e2b None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:04.319298 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910ed2f-2fe3-4180-9476-bd6f97142e2b None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:04.319298 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910ed2f-2fe3-4180-9476-bd6f97142e2b None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:04.320647 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 725/1471] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:35:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:35:04.326058 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7687e5e-b524-4f8f-9c1d-44d3785fbb8d None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:04.326494 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7687e5e-b524-4f8f-9c1d-44d3785fbb8d None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:04.326611 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7687e5e-b524-4f8f-9c1d-44d3785fbb8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:04.345197 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-f7687e5e-b524-4f8f-9c1d-44d3785fbb8d None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:35:04.346420 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-f7687e5e-b524-4f8f-9c1d-44d3785fbb8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:35:04.354960 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 747/1472] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:35:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:35:04.402732 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5beaaa-befb-4ff7-9aee-bdda261c2294 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:04.403025 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5beaaa-befb-4ff7-9aee-bdda261c2294 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:04.403302 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5beaaa-befb-4ff7-9aee-bdda261c2294 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:04.404658 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 726/1473] 10.222.0.13 () {54 vars in 962 bytes} [Fri Jun 24 08:35:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:35:04.410032 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9b714-8e5b-4f7b-bf48-e6bb366a3fb6 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:04.410323 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9b714-8e5b-4f7b-bf48-e6bb366a3fb6 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:04.410602 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9b714-8e5b-4f7b-bf48-e6bb366a3fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:04.428362 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-3cd9b714-8e5b-4f7b-bf48-e6bb366a3fb6 None None] MFA Rules not processed for user `d15c5f6aeb874191b73c8ee0410c6bc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:35:04.429554 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-3cd9b714-8e5b-4f7b-bf48-e6bb366a3fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:35:04.436105 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 748/1474] 10.222.0.13 () {58 vars in 1059 bytes} [Fri Jun 24 08:35:04 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:35:04.443422 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5beaaa-befb-4ff7-9aee-bdda261c2294 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:35:04.454239 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5beaaa-befb-4ff7-9aee-bdda261c2294 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:35:04.458379 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd201442-22d2-4b81-8619-233189487215 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd15c5f6aeb874191b73c8ee0410c6bc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:35:04.459008 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd201442-22d2-4b81-8619-233189487215 service glance] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:04.459290 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd201442-22d2-4b81-8619-233189487215 service glance] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:04.459564 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd201442-22d2-4b81-8619-233189487215 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:04.468905 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd201442-22d2-4b81-8619-233189487215 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:35:04.470378 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd201442-22d2-4b81-8619-233189487215 service glance] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:35:04.480686 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 727/1475] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:35:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:35:06.607991 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef7d460-5a19-4b8f-a9d0-3c4b5633cf47 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:06.608562 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef7d460-5a19-4b8f-a9d0-3c4b5633cf47 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:06.608562 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef7d460-5a19-4b8f-a9d0-3c4b5633cf47 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:06.610024 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 749/1476] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:35:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:35:06.615085 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8434c-c964-4763-8995-00c61abd1361 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:06.615538 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8434c-c964-4763-8995-00c61abd1361 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:06.615636 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8434c-c964-4763-8995-00c61abd1361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:06.633276 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-45c8434c-c964-4763-8995-00c61abd1361 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:35:06.634754 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-45c8434c-c964-4763-8995-00c61abd1361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:35:06.642500 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 728/1477] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:35:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:35:06.668171 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef7d460-5a19-4b8f-a9d0-3c4b5633cf47 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:35:06.680887 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef7d460-5a19-4b8f-a9d0-3c4b5633cf47 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:35:06.685287 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c519ee7-2d83-49d6-9ac0-15b66f6ebacc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:35:06.685976 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c519ee7-2d83-49d6-9ac0-15b66f6ebacc admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:06.686243 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c519ee7-2d83-49d6-9ac0-15b66f6ebacc admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:06.686515 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c519ee7-2d83-49d6-9ac0-15b66f6ebacc admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:06.687244 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c519ee7-2d83-49d6-9ac0-15b66f6ebacc admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:35:06.689019 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: WARNING py.warnings [None req-8c519ee7-2d83-49d6-9ac0-15b66f6ebacc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 08:35:06.689019 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: warnings.warn(msg) Jun 24 08:35:06.689019 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: Jun 24 08:35:06.690580 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c519ee7-2d83-49d6-9ac0-15b66f6ebacc admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:35:06.710077 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 750/1478] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Jun 24 08:35:06 2022] GET /identity/v3/projects => generated 2553 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:35:08.826963 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d1af9d-84db-4c87-be94-6370ed9223e3 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:08.827511 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d1af9d-84db-4c87-be94-6370ed9223e3 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:08.827511 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d1af9d-84db-4c87-be94-6370ed9223e3 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:08.828902 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 729/1479] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:35:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:35:08.835280 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c75417-21af-424f-a3b7-ea97217dc98b None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:08.835765 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c75417-21af-424f-a3b7-ea97217dc98b None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:08.835879 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c75417-21af-424f-a3b7-ea97217dc98b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:08.854484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-60c75417-21af-424f-a3b7-ea97217dc98b None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:35:08.855711 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-60c75417-21af-424f-a3b7-ea97217dc98b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:35:08.863857 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 751/1480] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:35:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:35:08.909259 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d1af9d-84db-4c87-be94-6370ed9223e3 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:35:08.923856 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d1af9d-84db-4c87-be94-6370ed9223e3 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:35:08.929208 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de52a364-3f32-4132-8d17-113c02269af8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74bb1bc6821b4500b26f160ccae5b7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:35:08.929893 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52a364-3f32-4132-8d17-113c02269af8 service nova] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:08.930177 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52a364-3f32-4132-8d17-113c02269af8 service nova] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:08.930453 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52a364-3f32-4132-8d17-113c02269af8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:08.939212 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de52a364-3f32-4132-8d17-113c02269af8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:35:08.941024 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de52a364-3f32-4132-8d17-113c02269af8 service nova] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:35:08.950931 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 730/1481] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:35:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:35:11.219971 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bbec8b-faae-498a-8ca6-f2c61bee5aac None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:11.220566 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bbec8b-faae-498a-8ca6-f2c61bee5aac None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:11.220566 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bbec8b-faae-498a-8ca6-f2c61bee5aac None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:11.222473 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 752/1482] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:35:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:35:11.227840 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ac7c2-f2ae-4486-acc6-9a967ea81af4 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:11.228503 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ac7c2-f2ae-4486-acc6-9a967ea81af4 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:11.228503 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ac7c2-f2ae-4486-acc6-9a967ea81af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:11.249017 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-cb6ac7c2-f2ae-4486-acc6-9a967ea81af4 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:35:11.250270 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-cb6ac7c2-f2ae-4486-acc6-9a967ea81af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:35:11.258332 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 731/1483] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:35:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:35:11.297408 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bbec8b-faae-498a-8ca6-f2c61bee5aac None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:35:11.309579 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bbec8b-faae-498a-8ca6-f2c61bee5aac None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:35:11.313165 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b67422-6265-4329-889c-bc2b0227cabe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:35:11.314099 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b67422-6265-4329-889c-bc2b0227cabe service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:11.315092 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b67422-6265-4329-889c-bc2b0227cabe service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:11.315481 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b67422-6265-4329-889c-bc2b0227cabe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:11.328039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b67422-6265-4329-889c-bc2b0227cabe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:35:11.329528 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b67422-6265-4329-889c-bc2b0227cabe service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:35:11.340326 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 753/1484] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:35:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:35:13.509132 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e12446-d4a8-48de-bc57-2a534950a5da None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:13.509714 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e12446-d4a8-48de-bc57-2a534950a5da None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:13.509714 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e12446-d4a8-48de-bc57-2a534950a5da None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:13.511203 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 732/1485] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:35:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:35:13.516454 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde5691-d85d-49c4-a103-4656d015b012 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:13.516886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde5691-d85d-49c4-a103-4656d015b012 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:13.517020 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde5691-d85d-49c4-a103-4656d015b012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:13.536235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-bfde5691-d85d-49c4-a103-4656d015b012 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:35:13.537457 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-bfde5691-d85d-49c4-a103-4656d015b012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:35:13.545459 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 754/1486] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:35:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:35:13.584257 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e12446-d4a8-48de-bc57-2a534950a5da None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:35:13.595075 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e12446-d4a8-48de-bc57-2a534950a5da None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:35:13.598669 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ccf30b-d483-4a56-9124-dfeda5b78171 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:35:13.599311 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ccf30b-d483-4a56-9124-dfeda5b78171 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:13.599593 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ccf30b-d483-4a56-9124-dfeda5b78171 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:13.599870 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ccf30b-d483-4a56-9124-dfeda5b78171 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:13.609286 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ccf30b-d483-4a56-9124-dfeda5b78171 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:35:13.610783 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ccf30b-d483-4a56-9124-dfeda5b78171 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:35:13.620568 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 733/1487] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:35:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:35:15.899531 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee0ed1-3361-4671-8345-dcf68c57fb42 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:15.900111 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee0ed1-3361-4671-8345-dcf68c57fb42 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:15.900111 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee0ed1-3361-4671-8345-dcf68c57fb42 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:15.901514 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 755/1488] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:35:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:35:15.906894 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d598f1c8-b132-4818-bf27-183348c05b0f None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:15.907335 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d598f1c8-b132-4818-bf27-183348c05b0f None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:15.907457 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d598f1c8-b132-4818-bf27-183348c05b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:15.925410 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-d598f1c8-b132-4818-bf27-183348c05b0f None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:35:15.926663 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d598f1c8-b132-4818-bf27-183348c05b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:35:15.935077 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 734/1489] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:35:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:35:15.977519 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbee0ed1-3361-4671-8345-dcf68c57fb42 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:35:15.988267 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbee0ed1-3361-4671-8345-dcf68c57fb42 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:35:15.991683 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cac85b7-872a-4a2b-9171-fde940cdf67b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:35:15.992316 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cac85b7-872a-4a2b-9171-fde940cdf67b service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:15.992596 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cac85b7-872a-4a2b-9171-fde940cdf67b service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:15.992893 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cac85b7-872a-4a2b-9171-fde940cdf67b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:16.003890 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cac85b7-872a-4a2b-9171-fde940cdf67b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:35:16.005589 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cac85b7-872a-4a2b-9171-fde940cdf67b service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:35:16.016154 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 756/1490] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:35:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:35:18.257375 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e51ff-37c2-4e90-b058-fb5552296427 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:18.257937 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e51ff-37c2-4e90-b058-fb5552296427 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:18.258058 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e51ff-37c2-4e90-b058-fb5552296427 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:18.259601 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 735/1491] 10.222.0.13 () {54 vars in 926 bytes} [Fri Jun 24 08:35:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:35:18.264977 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307bd1df-0d00-4d17-97c4-17a633338125 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:18.265465 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307bd1df-0d00-4d17-97c4-17a633338125 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:18.265577 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307bd1df-0d00-4d17-97c4-17a633338125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:18.284723 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-307bd1df-0d00-4d17-97c4-17a633338125 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:35:18.285947 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-307bd1df-0d00-4d17-97c4-17a633338125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:35:18.293713 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 757/1492] 10.222.0.13 () {58 vars in 1023 bytes} [Fri Jun 24 08:35:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:35:18.332781 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854e51ff-37c2-4e90-b058-fb5552296427 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:35:18.343694 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854e51ff-37c2-4e90-b058-fb5552296427 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:35:18.347882 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ce95e4-2156-4988-9979-5653a6cb3080 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:35:18.348521 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ce95e4-2156-4988-9979-5653a6cb3080 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:18.348799 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ce95e4-2156-4988-9979-5653a6cb3080 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:18.349066 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ce95e4-2156-4988-9979-5653a6cb3080 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:18.358846 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ce95e4-2156-4988-9979-5653a6cb3080 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:35:18.360308 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ce95e4-2156-4988-9979-5653a6cb3080 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:35:18.370969 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 736/1493] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:35:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:35:28.278759 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=33 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:35:28.366361 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=33 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:36:45.533801 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7285b8-9dd5-4cf9-9317-03c245b09ec1 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:36:45.534406 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7285b8-9dd5-4cf9-9317-03c245b09ec1 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:36:45.534406 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7285b8-9dd5-4cf9-9317-03c245b09ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:36:45.553269 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-1d7285b8-9dd5-4cf9-9317-03c245b09ec1 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:36:45.554573 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-1d7285b8-9dd5-4cf9-9317-03c245b09ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:36:45.563652 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 758/1494] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:36:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:36:45.572909 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec58f1-21f6-4659-8e0a-3a669120b33c None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:36:45.573435 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec58f1-21f6-4659-8e0a-3a669120b33c None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:36:45.573529 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec58f1-21f6-4659-8e0a-3a669120b33c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:36:45.591922 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-deec58f1-21f6-4659-8e0a-3a669120b33c None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:36:45.593128 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-deec58f1-21f6-4659-8e0a-3a669120b33c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:36:45.601012 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 737/1495] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:36:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:36:45.606772 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7285b8-9dd5-4cf9-9317-03c245b09ec1 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:36:45.616555 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7285b8-9dd5-4cf9-9317-03c245b09ec1 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:36:45.622717 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd480eda-cecb-445f-b6a0-3b710810ee61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:36:45.623375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd480eda-cecb-445f-b6a0-3b710810ee61 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:36:45.623710 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd480eda-cecb-445f-b6a0-3b710810ee61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:36:45.623940 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd480eda-cecb-445f-b6a0-3b710810ee61 admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:36:45.624665 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd480eda-cecb-445f-b6a0-3b710810ee61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:36:45.624973 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd480eda-cecb-445f-b6a0-3b710810ee61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:36:45.626015 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: WARNING py.warnings [None req-cd480eda-cecb-445f-b6a0-3b710810ee61 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 24 08:36:45.626015 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: warnings.warn(msg) Jun 24 08:36:45.626015 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: Jun 24 08:36:45.626886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd480eda-cecb-445f-b6a0-3b710810ee61 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:36:45.640043 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 759/1496] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:36:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:36:45.648002 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deec58f1-21f6-4659-8e0a-3a669120b33c None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:36:45.657268 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deec58f1-21f6-4659-8e0a-3a669120b33c None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:36:45.661806 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b129f-724b-4673-aae9-7cea94c14ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:36:45.662452 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b129f-724b-4673-aae9-7cea94c14ffe admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:36:45.662732 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b129f-724b-4673-aae9-7cea94c14ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:36:45.663007 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b129f-724b-4673-aae9-7cea94c14ffe admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:36:45.663681 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0b129f-724b-4673-aae9-7cea94c14ffe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:36:45.665483 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: WARNING py.warnings [None req-5a0b129f-724b-4673-aae9-7cea94c14ffe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 08:36:45.665483 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: warnings.warn(msg) Jun 24 08:36:45.665483 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: Jun 24 08:36:45.666690 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0b129f-724b-4673-aae9-7cea94c14ffe admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:36:45.705115 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 738/1497] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:36:45 2022] POST /identity/v3/projects => generated 379 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:36:45.710705 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd480eda-cecb-445f-b6a0-3b710810ee61 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:36:45.721229 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd480eda-cecb-445f-b6a0-3b710810ee61 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:36:45.777402 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466a970b-6879-4535-ad6f-54d42c674419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:36:45.778100 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a970b-6879-4535-ad6f-54d42c674419 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:36:45.778391 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a970b-6879-4535-ad6f-54d42c674419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:36:45.778669 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a970b-6879-4535-ad6f-54d42c674419 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:36:45.779345 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466a970b-6879-4535-ad6f-54d42c674419 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:36:45.780349 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: WARNING py.warnings [None req-466a970b-6879-4535-ad6f-54d42c674419 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 08:36:45.780349 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: warnings.warn(msg) Jun 24 08:36:45.780349 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: Jun 24 08:36:45.781560 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466a970b-6879-4535-ad6f-54d42c674419 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:36:45.800698 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 760/1498] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:36:45 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:36:45.804841 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b129f-724b-4673-aae9-7cea94c14ffe admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:36:45.815836 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b129f-724b-4673-aae9-7cea94c14ffe admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:36:45.820347 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdb593c-9822-4cf4-a0e5-e3b4fcbb7b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:36:45.821002 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb593c-9822-4cf4-a0e5-e3b4fcbb7b0f admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:36:45.821286 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb593c-9822-4cf4-a0e5-e3b4fcbb7b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:36:45.821562 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb593c-9822-4cf4-a0e5-e3b4fcbb7b0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:36:45.822323 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdb593c-9822-4cf4-a0e5-e3b4fcbb7b0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:36:45.823329 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: WARNING py.warnings [None req-7bdb593c-9822-4cf4-a0e5-e3b4fcbb7b0f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 24 08:36:45.823329 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: warnings.warn(msg) Jun 24 08:36:45.823329 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: Jun 24 08:36:45.824466 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdb593c-9822-4cf4-a0e5-e3b4fcbb7b0f admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:36:45.836963 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 739/1499] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:36:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:36:45.841422 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466a970b-6879-4535-ad6f-54d42c674419 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:36:45.851308 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466a970b-6879-4535-ad6f-54d42c674419 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:36:45.856564 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c35289-3c74-4a2d-9691-daf8e7170b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:36:45.857225 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c35289-3c74-4a2d-9691-daf8e7170b64 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:36:45.857507 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c35289-3c74-4a2d-9691-daf8e7170b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:36:45.857805 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c35289-3c74-4a2d-9691-daf8e7170b64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:36:45.858530 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c35289-3c74-4a2d-9691-daf8e7170b64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:36:45.859568 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: WARNING py.warnings [None req-e6c35289-3c74-4a2d-9691-daf8e7170b64 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 24 08:36:45.859568 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: warnings.warn(msg) Jun 24 08:36:45.859568 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: Jun 24 08:36:45.860460 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c35289-3c74-4a2d-9691-daf8e7170b64 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:36:45.875434 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 761/1500] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:36:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:36:45.879869 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdb593c-9822-4cf4-a0e5-e3b4fcbb7b0f admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:36:45.890973 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdb593c-9822-4cf4-a0e5-e3b4fcbb7b0f admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:36:45.895773 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40804a00-c5b5-45d2-a622-0599a1cd6d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:36:45.896675 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40804a00-c5b5-45d2-a622-0599a1cd6d52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:36:45.896957 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40804a00-c5b5-45d2-a622-0599a1cd6d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:36:45.897232 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40804a00-c5b5-45d2-a622-0599a1cd6d52 admin admin] PATH_INFO: `/v3/projects/b8a0af905a9f4071835563a004477981/users/ea027d4636c94c5790152a2241933011/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:36:45.913573 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40804a00-c5b5-45d2-a622-0599a1cd6d52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8a0af905a9f4071835563a004477981, user_id=ea027d4636c94c5790152a2241933011, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:36:45.914662 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: WARNING py.warnings [None req-40804a00-c5b5-45d2-a622-0599a1cd6d52 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 08:36:45.914662 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: warnings.warn(msg) Jun 24 08:36:45.914662 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: Jun 24 08:36:45.916601 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40804a00-c5b5-45d2-a622-0599a1cd6d52 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:36:45.933912 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 740/1501] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:36:45 2022] PUT /identity/v3/projects/b8a0af905a9f4071835563a004477981/users/ea027d4636c94c5790152a2241933011/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:36:45.946869 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a8ab0e-0f65-446e-8b1e-3c5af3fc4218 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:36:45.947228 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a8ab0e-0f65-446e-8b1e-3c5af3fc4218 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:36:45.947584 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a8ab0e-0f65-446e-8b1e-3c5af3fc4218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:36:45.974512 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-b5a8ab0e-0f65-446e-8b1e-3c5af3fc4218 None None] MFA Rules not processed for user `ea027d4636c94c5790152a2241933011`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:36:45.975729 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b5a8ab0e-0f65-446e-8b1e-3c5af3fc4218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:36:46.040772 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 762/1502] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:36:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:36:46.054572 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235575a-2b3c-42b7-ab26-fc6d48a75785 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:36:46.054863 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235575a-2b3c-42b7-ab26-fc6d48a75785 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:36:46.055143 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235575a-2b3c-42b7-ab26-fc6d48a75785 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:36:46.056530 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 741/1503] 10.222.0.13 () {54 vars in 963 bytes} [Fri Jun 24 08:36:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:36:46.061435 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ff8d6-918e-4b29-b610-3073d45c30fe None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:36:46.061768 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ff8d6-918e-4b29-b610-3073d45c30fe None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:36:46.062055 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ff8d6-918e-4b29-b610-3073d45c30fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:36:46.080633 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-746ff8d6-918e-4b29-b610-3073d45c30fe None None] MFA Rules not processed for user `44dcdfb5ca1541f9b4967a1f8ddbb34b`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:36:46.081891 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-746ff8d6-918e-4b29-b610-3073d45c30fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:36:46.125713 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:36:46.134711 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 763/1504] 10.222.0.13 () {58 vars in 1060 bytes} [Fri Jun 24 08:36:46 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:36:46.143593 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5235575a-2b3c-42b7-ab26-fc6d48a75785 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:36:46.155105 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5235575a-2b3c-42b7-ab26-fc6d48a75785 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:36:46.158248 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85a4f4b-e3c3-484d-a36d-1784892dd3f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:36:46.158883 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a4f4b-e3c3-484d-a36d-1784892dd3f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:36:46.159164 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a4f4b-e3c3-484d-a36d-1784892dd3f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:36:46.159438 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a4f4b-e3c3-484d-a36d-1784892dd3f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:36:46.168734 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85a4f4b-e3c3-484d-a36d-1784892dd3f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:36:46.170211 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85a4f4b-e3c3-484d-a36d-1784892dd3f7 service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:36:46.179400 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 742/1505] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:36:46 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:36:46.223063 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0807204-4b1c-4736-8f98-d62cd3c432e1 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:36:46.223491 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0807204-4b1c-4736-8f98-d62cd3c432e1 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:36:46.223776 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0807204-4b1c-4736-8f98-d62cd3c432e1 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:36:46.225051 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 764/1506] 10.222.0.13 () {52 vars in 829 bytes} [Fri Jun 24 08:36:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:36:46.228748 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85a4f4b-e3c3-484d-a36d-1784892dd3f7 service cinder] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:36:46.304827 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85a4f4b-e3c3-484d-a36d-1784892dd3f7 service cinder] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:36:46.310090 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01743586-cc6c-40db-9db0-3a77bdac6cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:36:46.310975 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01743586-cc6c-40db-9db0-3a77bdac6cdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:36:46.311259 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01743586-cc6c-40db-9db0-3a77bdac6cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:36:46.311536 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01743586-cc6c-40db-9db0-3a77bdac6cdc admin admin] PATH_INFO: `/v3/users/ea027d4636c94c5790152a2241933011` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:36:46.312748 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01743586-cc6c-40db-9db0-3a77bdac6cdc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea027d4636c94c5790152a2241933011)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:36:46.313808 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: WARNING py.warnings [None req-01743586-cc6c-40db-9db0-3a77bdac6cdc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 08:36:46.313808 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: warnings.warn(msg) Jun 24 08:36:46.313808 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: Jun 24 08:36:46.315019 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01743586-cc6c-40db-9db0-3a77bdac6cdc admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:36:46.392768 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-01743586-cc6c-40db-9db0-3a77bdac6cdc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea027d4636c94c5790152a2241933011'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:36:46.411619 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-01743586-cc6c-40db-9db0-3a77bdac6cdc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea027d4636c94c5790152a2241933011'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:36:46.412357 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-01743586-cc6c-40db-9db0-3a77bdac6cdc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea027d4636c94c5790152a2241933011'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:36:46.424577 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-01743586-cc6c-40db-9db0-3a77bdac6cdc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea027d4636c94c5790152a2241933011'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:36:46.425365 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-01743586-cc6c-40db-9db0-3a77bdac6cdc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea027d4636c94c5790152a2241933011'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:36:46.429372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:36:46.442166 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 743/1507] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:36:46 2022] DELETE /identity/v3/users/ea027d4636c94c5790152a2241933011 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:36:46.447199 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cafd2ef-ef4d-4f49-9a91-8ba893111aaf None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:36:46.447494 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cafd2ef-ef4d-4f49-9a91-8ba893111aaf None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:36:46.447893 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cafd2ef-ef4d-4f49-9a91-8ba893111aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:36:46.469292 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-4cafd2ef-ef4d-4f49-9a91-8ba893111aaf None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:36:46.470517 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-4cafd2ef-ef4d-4f49-9a91-8ba893111aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:36:46.530823 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 765/1508] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:36:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:36:46.614317 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01743586-cc6c-40db-9db0-3a77bdac6cdc admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:36:46.615818 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-01743586-cc6c-40db-9db0-3a77bdac6cdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:36:46.679564 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01743586-cc6c-40db-9db0-3a77bdac6cdc admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:36:46.683690 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754c6d4d-35a4-4e0d-a300-72852ae384d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:36:46.684347 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754c6d4d-35a4-4e0d-a300-72852ae384d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:36:46.684634 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754c6d4d-35a4-4e0d-a300-72852ae384d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:36:46.684913 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754c6d4d-35a4-4e0d-a300-72852ae384d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:36:46.694304 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754c6d4d-35a4-4e0d-a300-72852ae384d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:36:46.695761 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754c6d4d-35a4-4e0d-a300-72852ae384d8 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:36:46.706174 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 744/1509] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:36:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:36:47.077384 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cafd2ef-ef4d-4f49-9a91-8ba893111aaf None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:36:47.078937 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-4cafd2ef-ef4d-4f49-9a91-8ba893111aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:36:47.093884 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cafd2ef-ef4d-4f49-9a91-8ba893111aaf None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:36:47.099340 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8641a23a-17f4-4cdc-a3cd-078734fce623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:36:47.100207 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8641a23a-17f4-4cdc-a3cd-078734fce623 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:36:47.101316 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8641a23a-17f4-4cdc-a3cd-078734fce623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:36:47.101597 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8641a23a-17f4-4cdc-a3cd-078734fce623 admin admin] PATH_INFO: `/v3/projects/b8a0af905a9f4071835563a004477981` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:36:47.103177 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8641a23a-17f4-4cdc-a3cd-078734fce623 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8a0af905a9f4071835563a004477981)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:36:47.104346 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: WARNING py.warnings [None req-8641a23a-17f4-4cdc-a3cd-078734fce623 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 08:36:47.104346 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: warnings.warn(msg) Jun 24 08:36:47.104346 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: Jun 24 08:36:47.105932 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8641a23a-17f4-4cdc-a3cd-078734fce623 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:36:47.172124 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-8641a23a-17f4-4cdc-a3cd-078734fce623 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8a0af905a9f4071835563a004477981', 'request_id': 'req-8641a23a-17f4-4cdc-a3cd-078734fce623'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:36:47.186163 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-8641a23a-17f4-4cdc-a3cd-078734fce623 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8a0af905a9f4071835563a004477981', 'request_id': 'req-8641a23a-17f4-4cdc-a3cd-078734fce623'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:36:47.192327 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-8641a23a-17f4-4cdc-a3cd-078734fce623 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8a0af905a9f4071835563a004477981', 'request_id': 'req-8641a23a-17f4-4cdc-a3cd-078734fce623'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:36:47.207124 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-8641a23a-17f4-4cdc-a3cd-078734fce623 admin admin] The token cache is being invalidate because project b8a0af905a9f4071835563a004477981 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:36:47.207343 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-8641a23a-17f4-4cdc-a3cd-078734fce623 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:36:47.207965 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-8641a23a-17f4-4cdc-a3cd-078734fce623 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:36:47.209628 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 766/1510] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:36:47 2022] DELETE /identity/v3/projects/b8a0af905a9f4071835563a004477981 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:36:56.701766 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:36:57.203519 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:38:29.373919 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb29c9-91ec-43d1-9e4b-c471e93a048e None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:38:29.374369 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb29c9-91ec-43d1-9e4b-c471e93a048e None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:38:29.374747 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb29c9-91ec-43d1-9e4b-c471e93a048e None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:38:29.376110 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 745/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 08:38:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:38:29.380948 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7204a-a363-442a-a9ca-6ebee4d2df41 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:38:29.381408 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7204a-a363-442a-a9ca-6ebee4d2df41 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:38:29.381513 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7204a-a363-442a-a9ca-6ebee4d2df41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:38:29.402124 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-09d7204a-a363-442a-a9ca-6ebee4d2df41 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:38:29.403344 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-09d7204a-a363-442a-a9ca-6ebee4d2df41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:38:29.465227 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 767/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 08:38:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:38:29.622830 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb29c9-91ec-43d1-9e4b-c471e93a048e None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:38:29.624885 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-b1eb29c9-91ec-43d1-9e4b-c471e93a048e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:38:29.706628 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb29c9-91ec-43d1-9e4b-c471e93a048e None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:38:29.721878 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b242ae6a-a51b-4aae-b87a-7fd4b836c5a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74bb1bc6821b4500b26f160ccae5b7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:38:29.722609 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b242ae6a-a51b-4aae-b87a-7fd4b836c5a0 service nova] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:38:29.722878 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b242ae6a-a51b-4aae-b87a-7fd4b836c5a0 service nova] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:38:29.723155 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b242ae6a-a51b-4aae-b87a-7fd4b836c5a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:38:29.734478 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b242ae6a-a51b-4aae-b87a-7fd4b836c5a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:38:29.736526 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b242ae6a-a51b-4aae-b87a-7fd4b836c5a0 service nova] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:38:29.747434 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 746/1513] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:38:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:38:30.576736 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f02048-527d-4727-8f5b-53a183de3ea4 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:38:30.577264 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f02048-527d-4727-8f5b-53a183de3ea4 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:38:30.577264 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f02048-527d-4727-8f5b-53a183de3ea4 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:38:30.578614 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 768/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 08:38:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:38:30.583333 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c8eb3-a6b3-4b98-bf1a-fa9eedc4178f None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:38:30.583736 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c8eb3-a6b3-4b98-bf1a-fa9eedc4178f None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:38:30.583886 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c8eb3-a6b3-4b98-bf1a-fa9eedc4178f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:38:30.602155 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-a29c8eb3-a6b3-4b98-bf1a-fa9eedc4178f None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:38:30.603316 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-a29c8eb3-a6b3-4b98-bf1a-fa9eedc4178f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:38:30.612091 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 747/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 08:38:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:38:30.764805 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f02048-527d-4727-8f5b-53a183de3ea4 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:38:30.779730 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f02048-527d-4727-8f5b-53a183de3ea4 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:38:30.785089 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f74c8d7-f2cd-4c3e-aead-8bdac5667379 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74bb1bc6821b4500b26f160ccae5b7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:38:30.785764 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74c8d7-f2cd-4c3e-aead-8bdac5667379 service nova] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:38:30.786049 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74c8d7-f2cd-4c3e-aead-8bdac5667379 service nova] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:38:30.786321 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74c8d7-f2cd-4c3e-aead-8bdac5667379 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:38:30.795201 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f74c8d7-f2cd-4c3e-aead-8bdac5667379 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:38:30.796776 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f74c8d7-f2cd-4c3e-aead-8bdac5667379 service nova] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:38:30.806176 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 769/1516] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:38:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:38:31.407598 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36565c68-acd6-4348-b23c-361d6e422eab None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:38:31.407884 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36565c68-acd6-4348-b23c-361d6e422eab None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:38:31.408161 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36565c68-acd6-4348-b23c-361d6e422eab None None] PATH_INFO: `/v3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:38:31.409475 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 748/1517] 10.222.0.49 () {54 vars in 925 bytes} [Fri Jun 24 08:38:31 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:38:31.416093 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ede397-b27d-4b43-97be-b5787dac5dce None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:38:31.416380 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ede397-b27d-4b43-97be-b5787dac5dce None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:38:31.416659 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ede397-b27d-4b43-97be-b5787dac5dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:38:31.436807 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-d0ede397-b27d-4b43-97be-b5787dac5dce None None] MFA Rules not processed for user `39cc268bad604dd08594e3b39c7ecf5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:38:31.438058 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-d0ede397-b27d-4b43-97be-b5787dac5dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:38:31.510150 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 770/1518] 10.222.0.49 () {58 vars in 1015 bytes} [Fri Jun 24 08:38:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:38:40.245498 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728f876-3756-497d-8184-f30317b5f8f1 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:38:40.246073 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728f876-3756-497d-8184-f30317b5f8f1 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:38:40.246073 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728f876-3756-497d-8184-f30317b5f8f1 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:38:40.247517 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 749/1519] 10.222.0.13 () {54 vars in 960 bytes} [Fri Jun 24 08:38:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:38:40.253128 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f07b6-67a4-4a92-b592-d3e2254d9d65 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:38:40.253679 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f07b6-67a4-4a92-b592-d3e2254d9d65 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:38:40.253679 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f07b6-67a4-4a92-b592-d3e2254d9d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:38:40.272936 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-9a3f07b6-67a4-4a92-b592-d3e2254d9d65 None None] MFA Rules not processed for user `39cc268bad604dd08594e3b39c7ecf5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:38:40.274589 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-9a3f07b6-67a4-4a92-b592-d3e2254d9d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:38:40.283431 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 771/1520] 10.222.0.13 () {58 vars in 1057 bytes} [Fri Jun 24 08:38:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:38:40.291888 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e728f876-3756-497d-8184-f30317b5f8f1 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:38:40.306715 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e728f876-3756-497d-8184-f30317b5f8f1 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:38:40.311723 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f8aec1-ddb8-48f6-896d-b234a6409936 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39cc268bad604dd08594e3b39c7ecf5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:38:40.312362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8aec1-ddb8-48f6-896d-b234a6409936 service placement] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:38:40.312645 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8aec1-ddb8-48f6-896d-b234a6409936 service placement] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:38:40.312921 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8aec1-ddb8-48f6-896d-b234a6409936 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:38:40.321861 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f8aec1-ddb8-48f6-896d-b234a6409936 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:38:40.323462 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f8aec1-ddb8-48f6-896d-b234a6409936 service placement] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:38:40.333069 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 750/1521] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:38:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:38:42.006635 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9413ca-308c-43da-9994-4c21940549f6 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:38:42.007216 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9413ca-308c-43da-9994-4c21940549f6 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:38:42.007216 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9413ca-308c-43da-9994-4c21940549f6 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:38:42.008663 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 772/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 08:38:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:38:42.013497 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac360b-a1c4-4f80-b61c-672136305de4 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:38:42.013977 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac360b-a1c4-4f80-b61c-672136305de4 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:38:42.014124 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac360b-a1c4-4f80-b61c-672136305de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:38:42.034471 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-0fac360b-a1c4-4f80-b61c-672136305de4 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:38:42.035693 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-0fac360b-a1c4-4f80-b61c-672136305de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:38:42.044294 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 751/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 08:38:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:38:42.208840 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9413ca-308c-43da-9994-4c21940549f6 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:38:42.224717 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9413ca-308c-43da-9994-4c21940549f6 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:38:42.231088 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3445ff33-6f84-4828-8146-e84e21cf244c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74bb1bc6821b4500b26f160ccae5b7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:38:42.231743 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3445ff33-6f84-4828-8146-e84e21cf244c service nova] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:38:42.232029 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3445ff33-6f84-4828-8146-e84e21cf244c service nova] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:38:42.232308 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3445ff33-6f84-4828-8146-e84e21cf244c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:38:42.240972 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3445ff33-6f84-4828-8146-e84e21cf244c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:38:42.242861 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3445ff33-6f84-4828-8146-e84e21cf244c service nova] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:38:42.253384 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 773/1524] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:38:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:38:43.120804 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f6159-3395-4458-9d5d-4b1a392641ca None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:38:43.121351 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f6159-3395-4458-9d5d-4b1a392641ca None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:38:43.121351 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f6159-3395-4458-9d5d-4b1a392641ca None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:38:43.122766 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 752/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 08:38:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:38:43.128014 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b50cdab-e323-4b3b-a5b5-31e757746797 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:38:43.128461 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b50cdab-e323-4b3b-a5b5-31e757746797 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:38:43.128576 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b50cdab-e323-4b3b-a5b5-31e757746797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:38:43.147298 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-0b50cdab-e323-4b3b-a5b5-31e757746797 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:38:43.149064 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-0b50cdab-e323-4b3b-a5b5-31e757746797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:38:43.156740 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 774/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 08:38:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:38:43.308679 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0f6159-3395-4458-9d5d-4b1a392641ca None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:38:43.323476 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0f6159-3395-4458-9d5d-4b1a392641ca None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:38:43.328770 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a0f4b-1c2b-4c44-92fd-398a9dc105c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74bb1bc6821b4500b26f160ccae5b7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:38:43.329406 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a0f4b-1c2b-4c44-92fd-398a9dc105c6 service nova] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:38:43.329706 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a0f4b-1c2b-4c44-92fd-398a9dc105c6 service nova] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:38:43.329980 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a0f4b-1c2b-4c44-92fd-398a9dc105c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:38:43.339195 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56a0f4b-1c2b-4c44-92fd-398a9dc105c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:38:43.340776 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56a0f4b-1c2b-4c44-92fd-398a9dc105c6 service nova] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:38:43.350357 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 753/1527] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:38:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:38:47.363031 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b787e6-e870-43cb-9080-67daf3b1ed09 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:38:47.363600 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b787e6-e870-43cb-9080-67daf3b1ed09 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:38:47.363600 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b787e6-e870-43cb-9080-67daf3b1ed09 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:38:47.364940 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 775/1528] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 08:38:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:38:47.371235 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3d21a-967e-4ae4-8e46-4d3ec953b88a None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:38:47.371743 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3d21a-967e-4ae4-8e46-4d3ec953b88a None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:38:47.371839 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3d21a-967e-4ae4-8e46-4d3ec953b88a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:38:47.390194 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-e8f3d21a-967e-4ae4-8e46-4d3ec953b88a None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:38:47.391377 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-e8f3d21a-967e-4ae4-8e46-4d3ec953b88a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:38:47.399065 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 754/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 08:38:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:38:47.435693 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b787e6-e870-43cb-9080-67daf3b1ed09 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:38:47.437403 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-f1b787e6-e870-43cb-9080-67daf3b1ed09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:38:47.502300 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b787e6-e870-43cb-9080-67daf3b1ed09 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:38:47.505950 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e7638e-260a-4f98-8e68-04ee88c296c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:38:47.506611 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e7638e-260a-4f98-8e68-04ee88c296c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:38:47.506893 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e7638e-260a-4f98-8e68-04ee88c296c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:38:47.507243 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e7638e-260a-4f98-8e68-04ee88c296c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:38:47.517152 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e7638e-260a-4f98-8e68-04ee88c296c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:38:47.518598 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e7638e-260a-4f98-8e68-04ee88c296c0 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:38:47.523719 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:38:47.529205 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 776/1530] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:38:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:38:49.566925 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b0ec16-d9c8-42be-a52c-0bd3183beb5e None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:38:49.567461 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b0ec16-d9c8-42be-a52c-0bd3183beb5e None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:38:49.567461 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b0ec16-d9c8-42be-a52c-0bd3183beb5e None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:38:49.568851 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 755/1531] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 08:38:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:38:49.574409 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe6ccdb-f9ad-40c4-b326-89e4f241ee74 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:38:49.574870 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe6ccdb-f9ad-40c4-b326-89e4f241ee74 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:38:49.574972 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe6ccdb-f9ad-40c4-b326-89e4f241ee74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:38:49.594206 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-efe6ccdb-f9ad-40c4-b326-89e4f241ee74 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:38:49.595415 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-efe6ccdb-f9ad-40c4-b326-89e4f241ee74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:38:49.604707 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 777/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 08:38:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:38:49.641194 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b0ec16-d9c8-42be-a52c-0bd3183beb5e None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:38:49.652684 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b0ec16-d9c8-42be-a52c-0bd3183beb5e None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:38:49.656181 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570d001e-1eeb-4825-a667-19bd92604234 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:38:49.656825 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d001e-1eeb-4825-a667-19bd92604234 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:38:49.657105 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d001e-1eeb-4825-a667-19bd92604234 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:38:49.657388 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d001e-1eeb-4825-a667-19bd92604234 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:38:49.667732 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570d001e-1eeb-4825-a667-19bd92604234 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:38:49.669204 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570d001e-1eeb-4825-a667-19bd92604234 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:38:49.680093 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 756/1533] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:38:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:38:53.657754 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf27db-da72-4eb0-95b7-d21521f4a1ab None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:38:53.658285 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf27db-da72-4eb0-95b7-d21521f4a1ab None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:38:53.658285 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf27db-da72-4eb0-95b7-d21521f4a1ab None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:38:53.659616 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 778/1534] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 08:38:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:38:53.664700 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a022e18-e10c-4d1a-9309-a3a58b1f0d6b None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:38:53.665149 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a022e18-e10c-4d1a-9309-a3a58b1f0d6b None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:38:53.665253 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a022e18-e10c-4d1a-9309-a3a58b1f0d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:38:53.683965 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-7a022e18-e10c-4d1a-9309-a3a58b1f0d6b None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:38:53.685169 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-7a022e18-e10c-4d1a-9309-a3a58b1f0d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:38:53.692943 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 757/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 08:38:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:38:54.002873 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbf27db-da72-4eb0-95b7-d21521f4a1ab None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:38:54.004716 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-ecbf27db-da72-4eb0-95b7-d21521f4a1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:38:54.063279 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbf27db-da72-4eb0-95b7-d21521f4a1ab None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:38:54.066800 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda5ad59-40ad-4b11-a10d-6258dcd8ed9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:38:54.067470 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5ad59-40ad-4b11-a10d-6258dcd8ed9a service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:38:54.067752 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5ad59-40ad-4b11-a10d-6258dcd8ed9a service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:38:54.068166 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5ad59-40ad-4b11-a10d-6258dcd8ed9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:38:54.078027 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda5ad59-40ad-4b11-a10d-6258dcd8ed9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:38:54.079447 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda5ad59-40ad-4b11-a10d-6258dcd8ed9a service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:38:54.089168 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 779/1536] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:38:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:38:56.177194 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef3d76-0b6a-43e3-bbfb-ce1b40281ef8 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:38:56.177839 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef3d76-0b6a-43e3-bbfb-ce1b40281ef8 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:38:56.177839 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef3d76-0b6a-43e3-bbfb-ce1b40281ef8 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:38:56.179181 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 758/1537] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 08:38:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:38:56.185389 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5813675-88f0-4a6f-b952-0decbe5063c9 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:38:56.185904 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5813675-88f0-4a6f-b952-0decbe5063c9 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:38:56.186133 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5813675-88f0-4a6f-b952-0decbe5063c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:38:56.208964 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-b5813675-88f0-4a6f-b952-0decbe5063c9 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:38:56.208964 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b5813675-88f0-4a6f-b952-0decbe5063c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:38:56.218731 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 780/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 08:38:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:38:56.524468 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef3d76-0b6a-43e3-bbfb-ce1b40281ef8 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:38:56.526171 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-8bef3d76-0b6a-43e3-bbfb-ce1b40281ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:38:56.539828 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef3d76-0b6a-43e3-bbfb-ce1b40281ef8 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:38:56.543381 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e183c5c-4c7f-4e0d-ae18-778647b0516e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:38:56.544012 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e183c5c-4c7f-4e0d-ae18-778647b0516e service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:38:56.544295 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e183c5c-4c7f-4e0d-ae18-778647b0516e service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:38:56.544571 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e183c5c-4c7f-4e0d-ae18-778647b0516e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:38:56.553468 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e183c5c-4c7f-4e0d-ae18-778647b0516e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:38:56.555478 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e183c5c-4c7f-4e0d-ae18-778647b0516e service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:38:56.564771 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 759/1539] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:38:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:39:06.201040 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=32 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:39:06.560687 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=70 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:26.943876 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d8cd6-1ab8-45bf-9a57-a32ae205c55f None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:26.945507 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d8cd6-1ab8-45bf-9a57-a32ae205c55f None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:26.946066 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d8cd6-1ab8-45bf-9a57-a32ae205c55f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:26.979466 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-4b4d8cd6-1ab8-45bf-9a57-a32ae205c55f None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:26.981464 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-4b4d8cd6-1ab8-45bf-9a57-a32ae205c55f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:26.994364 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 781/1540] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:40:27.004988 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52149580-14f3-40e8-be95-8115ed0731f4 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:27.006193 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52149580-14f3-40e8-be95-8115ed0731f4 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:27.007531 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52149580-14f3-40e8-be95-8115ed0731f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:27.034921 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-52149580-14f3-40e8-be95-8115ed0731f4 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:27.036702 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-52149580-14f3-40e8-be95-8115ed0731f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:27.047709 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 760/1541] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:27.052539 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4d8cd6-1ab8-45bf-9a57-a32ae205c55f None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:27.066372 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4d8cd6-1ab8-45bf-9a57-a32ae205c55f None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:27.076295 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545e5dcd-08e2-40dd-addd-989b13956c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:27.077532 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e5dcd-08e2-40dd-addd-989b13956c47 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:27.078058 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e5dcd-08e2-40dd-addd-989b13956c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:27.078818 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e5dcd-08e2-40dd-addd-989b13956c47 admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:27.079821 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545e5dcd-08e2-40dd-addd-989b13956c47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:40:27.080374 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545e5dcd-08e2-40dd-addd-989b13956c47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:27.082747 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545e5dcd-08e2-40dd-addd-989b13956c47 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:27.098802 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f8e92-04dc-45e8-9a2e-decafeb955cc None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:27.099060 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 782/1542] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:40:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:40:27.100678 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f8e92-04dc-45e8-9a2e-decafeb955cc None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:27.101326 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f8e92-04dc-45e8-9a2e-decafeb955cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:27.107619 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545e5dcd-08e2-40dd-addd-989b13956c47 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:27.118471 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545e5dcd-08e2-40dd-addd-989b13956c47 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:27.133039 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-7d6f8e92-04dc-45e8-9a2e-decafeb955cc None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:27.134740 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-7d6f8e92-04dc-45e8-9a2e-decafeb955cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:27.135290 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb73db47-39fe-40d1-8431-b26469d39990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:27.135985 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb73db47-39fe-40d1-8431-b26469d39990 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:27.136261 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb73db47-39fe-40d1-8431-b26469d39990 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:27.136537 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb73db47-39fe-40d1-8431-b26469d39990 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:27.138037 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb73db47-39fe-40d1-8431-b26469d39990 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:27.138282 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: WARNING py.warnings [None req-eb73db47-39fe-40d1-8431-b26469d39990 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 08:40:27.138282 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: warnings.warn(msg) Jun 24 08:40:27.138282 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: Jun 24 08:40:27.139498 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb73db47-39fe-40d1-8431-b26469d39990 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:27.153806 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 761/1543] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:40:27.165050 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d7505-52ba-4a57-89da-3385ed9b43e5 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:27.165760 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d7505-52ba-4a57-89da-3385ed9b43e5 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:27.166273 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d7505-52ba-4a57-89da-3385ed9b43e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:27.198571 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 783/1544] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:40:27 2022] POST /identity/v3/projects => generated 403 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:40:27.199857 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ee5bb-ec3c-4868-9557-7548339d393c None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:27.200265 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ee5bb-ec3c-4868-9557-7548339d393c None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:27.200572 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ee5bb-ec3c-4868-9557-7548339d393c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:27.205780 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-9b5d7505-52ba-4a57-89da-3385ed9b43e5 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:27.207731 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-9b5d7505-52ba-4a57-89da-3385ed9b43e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:27.236401 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-ef9ee5bb-ec3c-4868-9557-7548339d393c None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:27.237711 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-ef9ee5bb-ec3c-4868-9557-7548339d393c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:27.295932 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 762/1545] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:27.298960 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27198b87-79cf-46be-9bec-7cd2d87d2368 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:27.299840 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27198b87-79cf-46be-9bec-7cd2d87d2368 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:27.300378 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27198b87-79cf-46be-9bec-7cd2d87d2368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:27.341007 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-27198b87-79cf-46be-9bec-7cd2d87d2368 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:27.342301 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-27198b87-79cf-46be-9bec-7cd2d87d2368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:27.351474 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 763/1546] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:27.352325 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27198b87-79cf-46be-9bec-7cd2d87d2368 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:27.370047 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27198b87-79cf-46be-9bec-7cd2d87d2368 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:27.380782 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2cf95c-ae8f-4c53-9c6c-51a82c154da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:27.381629 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2cf95c-ae8f-4c53-9c6c-51a82c154da9 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:27.382717 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2cf95c-ae8f-4c53-9c6c-51a82c154da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:27.382999 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2cf95c-ae8f-4c53-9c6c-51a82c154da9 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:27.384019 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2cf95c-ae8f-4c53-9c6c-51a82c154da9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:27.385098 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: WARNING py.warnings [None req-6d2cf95c-ae8f-4c53-9c6c-51a82c154da9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 08:40:27.385098 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: warnings.warn(msg) Jun 24 08:40:27.385098 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: Jun 24 08:40:27.386399 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2cf95c-ae8f-4c53-9c6c-51a82c154da9 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:27.399140 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 784/1547] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:27.401463 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae25a27-b28c-461d-87e6-f46350f599eb None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:27.402431 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae25a27-b28c-461d-87e6-f46350f599eb None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:27.402831 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae25a27-b28c-461d-87e6-f46350f599eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:27.411159 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 764/1548] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:40:27 2022] POST /identity/v3/users => generated 302 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:40:27.412293 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2cf95c-ae8f-4c53-9c6c-51a82c154da9 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:27.422906 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2cf95c-ae8f-4c53-9c6c-51a82c154da9 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:27.427582 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe82e0d-0ede-4e06-84f7-93028148fced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:27.428812 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe82e0d-0ede-4e06-84f7-93028148fced admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:27.429094 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe82e0d-0ede-4e06-84f7-93028148fced admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:27.429368 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe82e0d-0ede-4e06-84f7-93028148fced admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:27.429642 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-3ae25a27-b28c-461d-87e6-f46350f599eb None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:27.430124 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe82e0d-0ede-4e06-84f7-93028148fced admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:40:27.430426 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe82e0d-0ede-4e06-84f7-93028148fced admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:27.431192 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-3ae25a27-b28c-461d-87e6-f46350f599eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:27.431451 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: WARNING py.warnings [None req-2fe82e0d-0ede-4e06-84f7-93028148fced admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 24 08:40:27.431451 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: warnings.warn(msg) Jun 24 08:40:27.431451 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: Jun 24 08:40:27.432376 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe82e0d-0ede-4e06-84f7-93028148fced admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:27.445723 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 785/1549] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:27.445723 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5463b8-8ceb-416c-a085-85177d470169 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:27.445723 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5463b8-8ceb-416c-a085-85177d470169 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:27.445723 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5463b8-8ceb-416c-a085-85177d470169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:27.458172 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 765/1550] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:40:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:40:27.458172 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b9e2a-a20c-498b-a7a5-25548271bc64 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:27.458172 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b9e2a-a20c-498b-a7a5-25548271bc64 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:27.458172 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b9e2a-a20c-498b-a7a5-25548271bc64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:27.470688 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-9e5463b8-8ceb-416c-a085-85177d470169 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:27.471904 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-9e5463b8-8ceb-416c-a085-85177d470169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:27.476723 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-0e1b9e2a-a20c-498b-a7a5-25548271bc64 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:27.479032 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-0e1b9e2a-a20c-498b-a7a5-25548271bc64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:27.489714 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 786/1551] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:27.489714 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174960e7-c769-48b5-ab49-24f9f0630b31 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:27.489714 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174960e7-c769-48b5-ab49-24f9f0630b31 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:27.489714 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174960e7-c769-48b5-ab49-24f9f0630b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:27.499140 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 766/1552] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:27.500506 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f680fb-e92f-4862-bfb7-61e9d773eb62 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:27.500868 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f680fb-e92f-4862-bfb7-61e9d773eb62 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:27.501211 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f680fb-e92f-4862-bfb7-61e9d773eb62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:27.510431 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-174960e7-c769-48b5-ab49-24f9f0630b31 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:27.511644 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-174960e7-c769-48b5-ab49-24f9f0630b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:27.520541 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-b4f680fb-e92f-4862-bfb7-61e9d773eb62 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:27.521600 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 787/1553] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:27.522361 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-b4f680fb-e92f-4862-bfb7-61e9d773eb62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:27.523237 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174960e7-c769-48b5-ab49-24f9f0630b31 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:27.532504 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 767/1554] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:27.533980 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18660e5f-f9fc-4c58-9e13-85c82d751ea3 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:27.534280 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18660e5f-f9fc-4c58-9e13-85c82d751ea3 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:27.534551 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18660e5f-f9fc-4c58-9e13-85c82d751ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:27.535692 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174960e7-c769-48b5-ab49-24f9f0630b31 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:27.542015 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e2bc6a-2c53-4c4d-aef8-f2c0a8fdb450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:27.542811 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e2bc6a-2c53-4c4d-aef8-f2c0a8fdb450 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:27.543275 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e2bc6a-2c53-4c4d-aef8-f2c0a8fdb450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:27.543558 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e2bc6a-2c53-4c4d-aef8-f2c0a8fdb450 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:27.544327 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e2bc6a-2c53-4c4d-aef8-f2c0a8fdb450 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:27.546031 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e2bc6a-2c53-4c4d-aef8-f2c0a8fdb450 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:27.553497 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-18660e5f-f9fc-4c58-9e13-85c82d751ea3 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:27.554673 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-18660e5f-f9fc-4c58-9e13-85c82d751ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:27.557220 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 788/1555] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:40:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:40:27.557915 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e2bc6a-2c53-4c4d-aef8-f2c0a8fdb450 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:27.562954 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 768/1556] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:27.564380 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204babd9-7e8d-49a5-80b5-2a8db2d3343a None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:27.564668 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204babd9-7e8d-49a5-80b5-2a8db2d3343a None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:27.564943 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204babd9-7e8d-49a5-80b5-2a8db2d3343a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:27.571412 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e2bc6a-2c53-4c4d-aef8-f2c0a8fdb450 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:27.576178 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0252acc6-5bad-4caf-96d3-fd2c43d84af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:27.576842 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252acc6-5bad-4caf-96d3-fd2c43d84af2 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:27.577604 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252acc6-5bad-4caf-96d3-fd2c43d84af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:27.577928 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252acc6-5bad-4caf-96d3-fd2c43d84af2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:27.578628 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0252acc6-5bad-4caf-96d3-fd2c43d84af2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:27.580546 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0252acc6-5bad-4caf-96d3-fd2c43d84af2 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:27.583387 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-204babd9-7e8d-49a5-80b5-2a8db2d3343a None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:27.585119 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-204babd9-7e8d-49a5-80b5-2a8db2d3343a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:27.594551 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 769/1557] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:27.595384 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204babd9-7e8d-49a5-80b5-2a8db2d3343a None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:27.607663 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204babd9-7e8d-49a5-80b5-2a8db2d3343a None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:27.612626 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e456ca-198c-43c0-a578-44815ef45ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:27.613457 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e456ca-198c-43c0-a578-44815ef45ead admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:27.613664 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 789/1558] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:40:27 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:40:27.613813 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e456ca-198c-43c0-a578-44815ef45ead admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:27.614174 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e456ca-198c-43c0-a578-44815ef45ead admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:27.614945 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e456ca-198c-43c0-a578-44815ef45ead admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:40:27.615065 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574fadd7-6318-4b78-a5a6-30c928693ae0 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:27.615251 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e456ca-198c-43c0-a578-44815ef45ead admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:27.615351 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574fadd7-6318-4b78-a5a6-30c928693ae0 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:27.615591 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574fadd7-6318-4b78-a5a6-30c928693ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:27.617111 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e456ca-198c-43c0-a578-44815ef45ead admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:27.629834 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 770/1559] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:40:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:40:27.631487 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bc8a44-b209-4804-99bd-49306d9810ff None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:27.632427 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bc8a44-b209-4804-99bd-49306d9810ff None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:27.632716 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bc8a44-b209-4804-99bd-49306d9810ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:27.634301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-574fadd7-6318-4b78-a5a6-30c928693ae0 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:27.635507 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-574fadd7-6318-4b78-a5a6-30c928693ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:27.652897 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-97bc8a44-b209-4804-99bd-49306d9810ff None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:27.654214 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-97bc8a44-b209-4804-99bd-49306d9810ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:27.700371 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 790/1560] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:27.701751 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574fadd7-6318-4b78-a5a6-30c928693ae0 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:27.714356 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574fadd7-6318-4b78-a5a6-30c928693ae0 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:27.718787 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173826f8-c70c-4d71-ae79-1c2bcb8e11f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:27.719447 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173826f8-c70c-4d71-ae79-1c2bcb8e11f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:27.719695 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 771/1561] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:27.719795 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173826f8-c70c-4d71-ae79-1c2bcb8e11f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:27.719988 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173826f8-c70c-4d71-ae79-1c2bcb8e11f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:27.720543 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bc8a44-b209-4804-99bd-49306d9810ff None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:27.720716 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173826f8-c70c-4d71-ae79-1c2bcb8e11f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:40:27.721090 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173826f8-c70c-4d71-ae79-1c2bcb8e11f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:27.722678 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173826f8-c70c-4d71-ae79-1c2bcb8e11f0 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:27.724412 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:27.732268 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bc8a44-b209-4804-99bd-49306d9810ff None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:27.744899 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 791/1562] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:40:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:40:27.744899 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173826f8-c70c-4d71-ae79-1c2bcb8e11f0 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:27.745232 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e957d69-69b3-4c19-a7b1-9eb69214a31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:27.745232 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e957d69-69b3-4c19-a7b1-9eb69214a31e admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:27.745232 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e957d69-69b3-4c19-a7b1-9eb69214a31e admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:27.745232 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e957d69-69b3-4c19-a7b1-9eb69214a31e admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:27.745583 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e957d69-69b3-4c19-a7b1-9eb69214a31e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:27.745583 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e957d69-69b3-4c19-a7b1-9eb69214a31e admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:27.748476 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:27.749176 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173826f8-c70c-4d71-ae79-1c2bcb8e11f0 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:27.753817 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 772/1563] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:40:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:40:27.754159 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd1962-1a56-4316-87e8-396edbf1470f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:27.754662 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd1962-1a56-4316-87e8-396edbf1470f admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:27.754786 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e957d69-69b3-4c19-a7b1-9eb69214a31e admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:27.754974 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd1962-1a56-4316-87e8-396edbf1470f admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:27.755229 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd1962-1a56-4316-87e8-396edbf1470f admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:27.756114 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd1962-1a56-4316-87e8-396edbf1470f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:40:27.756257 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd1962-1a56-4316-87e8-396edbf1470f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:27.757953 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd1962-1a56-4316-87e8-396edbf1470f admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:27.766461 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e957d69-69b3-4c19-a7b1-9eb69214a31e admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:27.771080 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb8f785-7280-4c88-b1bb-a484027aeb41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:27.778988 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 792/1564] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:40:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:40:27.779178 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8f785-7280-4c88-b1bb-a484027aeb41 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:27.779178 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8f785-7280-4c88-b1bb-a484027aeb41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:27.779178 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8f785-7280-4c88-b1bb-a484027aeb41 admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:27.779438 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd1962-1a56-4316-87e8-396edbf1470f admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:27.779534 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb8f785-7280-4c88-b1bb-a484027aeb41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:40:27.779534 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb8f785-7280-4c88-b1bb-a484027aeb41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:27.779534 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb8f785-7280-4c88-b1bb-a484027aeb41 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:27.789779 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd1962-1a56-4316-87e8-396edbf1470f admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:27.798864 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 773/1565] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:40:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:40:27.798864 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb8f785-7280-4c88-b1bb-a484027aeb41 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:27.799166 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f945fd-87aa-43b5-8ee1-5d93694b20de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:27.799166 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f945fd-87aa-43b5-8ee1-5d93694b20de admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:27.799166 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f945fd-87aa-43b5-8ee1-5d93694b20de admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:27.799166 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f945fd-87aa-43b5-8ee1-5d93694b20de admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:27.799531 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f945fd-87aa-43b5-8ee1-5d93694b20de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:27.799531 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f945fd-87aa-43b5-8ee1-5d93694b20de admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:27.802805 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb8f785-7280-4c88-b1bb-a484027aeb41 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:27.807589 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d5a93-e5e7-4f78-829b-8d37838b8ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:27.808252 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d5a93-e5e7-4f78-829b-8d37838b8ccf admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:27.808538 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d5a93-e5e7-4f78-829b-8d37838b8ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:27.808839 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d5a93-e5e7-4f78-829b-8d37838b8ccf admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:27.809617 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15d5a93-e5e7-4f78-829b-8d37838b8ccf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:27.811771 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15d5a93-e5e7-4f78-829b-8d37838b8ccf admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:27.820524 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 793/1566] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:40:27 2022] POST /identity/v3/users => generated 288 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:40:27.821942 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414940c-c62d-495d-b084-f7f7ddcf6504 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:27.822230 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414940c-c62d-495d-b084-f7f7ddcf6504 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:27.822507 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414940c-c62d-495d-b084-f7f7ddcf6504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:27.840827 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 774/1567] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:40:27 2022] POST /identity/v3/projects => generated 389 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:40:27.841479 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d5a93-e5e7-4f78-829b-8d37838b8ccf admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:27.844787 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-b414940c-c62d-495d-b084-f7f7ddcf6504 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:27.846020 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b414940c-c62d-495d-b084-f7f7ddcf6504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:27.909966 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d5a93-e5e7-4f78-829b-8d37838b8ccf admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:27.912021 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 794/1568] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:27.913047 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b414940c-c62d-495d-b084-f7f7ddcf6504 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:27.914956 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf69e8a-d7c7-4612-8f0e-26e2296b787e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:27.915660 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf69e8a-d7c7-4612-8f0e-26e2296b787e admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:27.915943 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf69e8a-d7c7-4612-8f0e-26e2296b787e admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:27.916212 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf69e8a-d7c7-4612-8f0e-26e2296b787e admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:27.916943 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf69e8a-d7c7-4612-8f0e-26e2296b787e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:40:27.917257 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf69e8a-d7c7-4612-8f0e-26e2296b787e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:27.918922 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf69e8a-d7c7-4612-8f0e-26e2296b787e admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:27.922746 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b414940c-c62d-495d-b084-f7f7ddcf6504 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:27.927307 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414e04a7-b4fa-42ec-b8c6-d2ae57417e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:27.927962 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e04a7-b4fa-42ec-b8c6-d2ae57417e6f admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:27.928300 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e04a7-b4fa-42ec-b8c6-d2ae57417e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:27.928575 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e04a7-b4fa-42ec-b8c6-d2ae57417e6f admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:27.929422 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414e04a7-b4fa-42ec-b8c6-d2ae57417e6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:27.938483 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 775/1569] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:40:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:40:27.938671 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414e04a7-b4fa-42ec-b8c6-d2ae57417e6f admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:27.938812 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf69e8a-d7c7-4612-8f0e-26e2296b787e admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:27.946283 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf69e8a-d7c7-4612-8f0e-26e2296b787e admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:27.951238 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5ca1da-e56b-4ba8-9d04-515e1781ebc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:27.952275 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ca1da-e56b-4ba8-9d04-515e1781ebc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:27.952664 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ca1da-e56b-4ba8-9d04-515e1781ebc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:27.952964 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ca1da-e56b-4ba8-9d04-515e1781ebc6 admin admin] PATH_INFO: `/v3/projects/35fe6df7ad774eb8ad6a7ccf008276cf/users/2be4a70caeb4445a90008b8213d6f8bf/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:27.970191 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 795/1570] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:40:27 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:40:27.971009 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414e04a7-b4fa-42ec-b8c6-d2ae57417e6f admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:27.971833 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5ca1da-e56b-4ba8-9d04-515e1781ebc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35fe6df7ad774eb8ad6a7ccf008276cf, user_id=2be4a70caeb4445a90008b8213d6f8bf, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:27.975127 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5ca1da-e56b-4ba8-9d04-515e1781ebc6 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:27.991710 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 776/1571] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:40:27 2022] PUT /identity/v3/projects/35fe6df7ad774eb8ad6a7ccf008276cf/users/2be4a70caeb4445a90008b8213d6f8bf/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:40:27.995008 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5ca1da-e56b-4ba8-9d04-515e1781ebc6 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:28.038261 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414e04a7-b4fa-42ec-b8c6-d2ae57417e6f admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:28.059491 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5ca1da-e56b-4ba8-9d04-515e1781ebc6 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:28.066971 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1258c3-8fc1-41da-a658-7d50380527ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:28.067647 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1258c3-8fc1-41da-a658-7d50380527ce admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:28.067926 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1258c3-8fc1-41da-a658-7d50380527ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:28.068234 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1258c3-8fc1-41da-a658-7d50380527ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:28.069414 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1258c3-8fc1-41da-a658-7d50380527ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:28.071358 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1258c3-8fc1-41da-a658-7d50380527ce admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:28.086506 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:28.099069 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 777/1572] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:40:27 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:40:28.100222 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1258c3-8fc1-41da-a658-7d50380527ce admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:28.100782 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773370b1-08c5-4421-9afb-846cd5c2fdcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:28.101446 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773370b1-08c5-4421-9afb-846cd5c2fdcd admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:28.101749 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773370b1-08c5-4421-9afb-846cd5c2fdcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:28.102132 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773370b1-08c5-4421-9afb-846cd5c2fdcd admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:28.102981 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773370b1-08c5-4421-9afb-846cd5c2fdcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:28.105134 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773370b1-08c5-4421-9afb-846cd5c2fdcd admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:28.125156 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:28.133060 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 796/1573] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:40:27 2022] POST /identity/v3/projects => generated 393 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:40:28.134055 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773370b1-08c5-4421-9afb-846cd5c2fdcd admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:28.172317 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1258c3-8fc1-41da-a658-7d50380527ce admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:28.199930 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773370b1-08c5-4421-9afb-846cd5c2fdcd admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:28.204619 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a5bbcb-36d7-4d97-b56e-b49a243235e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:28.205298 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a5bbcb-36d7-4d97-b56e-b49a243235e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:28.205578 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a5bbcb-36d7-4d97-b56e-b49a243235e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:28.205882 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a5bbcb-36d7-4d97-b56e-b49a243235e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:28.206575 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a5bbcb-36d7-4d97-b56e-b49a243235e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:28.208528 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a5bbcb-36d7-4d97-b56e-b49a243235e2 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:28.238476 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 797/1574] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:40:28 2022] POST /identity/v3/users => generated 295 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:40:28.239533 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a5bbcb-36d7-4d97-b56e-b49a243235e2 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:28.249328 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adf934f-7049-4445-8003-93acbac1c13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:28.250241 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adf934f-7049-4445-8003-93acbac1c13c admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:28.250549 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adf934f-7049-4445-8003-93acbac1c13c admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:28.250836 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adf934f-7049-4445-8003-93acbac1c13c admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:28.251608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adf934f-7049-4445-8003-93acbac1c13c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:28.253316 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adf934f-7049-4445-8003-93acbac1c13c admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:28.255230 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a5bbcb-36d7-4d97-b56e-b49a243235e2 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:28.263110 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d650c4af-5315-4898-8bdb-76a0785769f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:28.263783 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d650c4af-5315-4898-8bdb-76a0785769f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:28.264063 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d650c4af-5315-4898-8bdb-76a0785769f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:28.264338 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d650c4af-5315-4898-8bdb-76a0785769f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:28.265071 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d650c4af-5315-4898-8bdb-76a0785769f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:40:28.265379 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d650c4af-5315-4898-8bdb-76a0785769f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:28.265624 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 778/1575] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:40:28.266546 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adf934f-7049-4445-8003-93acbac1c13c admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:28.267029 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d650c4af-5315-4898-8bdb-76a0785769f8 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:28.277726 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adf934f-7049-4445-8003-93acbac1c13c admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:28.282483 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e977a2e-995a-4eed-859f-89f4f5192c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:28.283125 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e977a2e-995a-4eed-859f-89f4f5192c63 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:28.283432 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e977a2e-995a-4eed-859f-89f4f5192c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:28.283709 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e977a2e-995a-4eed-859f-89f4f5192c63 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:28.284229 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 798/1576] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:40:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:40:28.284398 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e977a2e-995a-4eed-859f-89f4f5192c63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:28.285047 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d650c4af-5315-4898-8bdb-76a0785769f8 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:28.286436 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e977a2e-995a-4eed-859f-89f4f5192c63 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:28.298712 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d650c4af-5315-4898-8bdb-76a0785769f8 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:28.303976 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbeecb3-d864-4f91-a09b-777de941ae29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:28.304669 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbeecb3-d864-4f91-a09b-777de941ae29 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:28.304932 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbeecb3-d864-4f91-a09b-777de941ae29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:28.305242 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbeecb3-d864-4f91-a09b-777de941ae29 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:28.305983 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbeecb3-d864-4f91-a09b-777de941ae29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:28.308345 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbeecb3-d864-4f91-a09b-777de941ae29 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:28.315778 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 779/1577] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:40:28 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:40:28.320542 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0389a5-4ec8-444f-837c-d351df8e07db None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:28.320861 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0389a5-4ec8-444f-837c-d351df8e07db None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:28.321301 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0389a5-4ec8-444f-837c-d351df8e07db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:28.331659 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 799/1578] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:40:28 2022] POST /identity/v3/users => generated 287 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:40:28.332468 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbeecb3-d864-4f91-a09b-777de941ae29 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:28.343838 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbeecb3-d864-4f91-a09b-777de941ae29 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:28.352143 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-6d0389a5-4ec8-444f-837c-d351df8e07db None None] MFA Rules not processed for user `2be4a70caeb4445a90008b8213d6f8bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:28.353709 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-6d0389a5-4ec8-444f-837c-d351df8e07db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:28.407767 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38876fe5-6f0e-4f10-8700-7ef31efc3edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:28.408443 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38876fe5-6f0e-4f10-8700-7ef31efc3edd admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:28.408722 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38876fe5-6f0e-4f10-8700-7ef31efc3edd admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:28.408995 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38876fe5-6f0e-4f10-8700-7ef31efc3edd admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:28.409676 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38876fe5-6f0e-4f10-8700-7ef31efc3edd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:28.411629 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38876fe5-6f0e-4f10-8700-7ef31efc3edd admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:28.430335 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 780/1579] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:28 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:28.431168 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0389a5-4ec8-444f-837c-d351df8e07db None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:28.435986 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 800/1580] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:40:28 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:40:28.435986 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38876fe5-6f0e-4f10-8700-7ef31efc3edd admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:28.442102 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0389a5-4ec8-444f-837c-d351df8e07db None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:28.444005 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38876fe5-6f0e-4f10-8700-7ef31efc3edd admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:28.447801 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7603d5f-ed45-4bf7-927a-45f25f289d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:28.448481 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7603d5f-ed45-4bf7-927a-45f25f289d5a admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:28.448766 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7603d5f-ed45-4bf7-927a-45f25f289d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:28.449068 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7603d5f-ed45-4bf7-927a-45f25f289d5a admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:28.449180 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a99c46-ece2-4c28-b4e4-9cefbd35d7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:28.449863 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7603d5f-ed45-4bf7-927a-45f25f289d5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:28.449966 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a99c46-ece2-4c28-b4e4-9cefbd35d7fc admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:28.450183 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a99c46-ece2-4c28-b4e4-9cefbd35d7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:28.450462 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a99c46-ece2-4c28-b4e4-9cefbd35d7fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:28.451212 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a99c46-ece2-4c28-b4e4-9cefbd35d7fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:28.452608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7603d5f-ed45-4bf7-927a-45f25f289d5a admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:28.452797 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a99c46-ece2-4c28-b4e4-9cefbd35d7fc admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:28.465019 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 801/1581] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:40:28.465937 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a99c46-ece2-4c28-b4e4-9cefbd35d7fc admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:28.475834 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 781/1582] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:40:28 2022] POST /identity/v3/users => generated 297 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:40:28.476715 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7603d5f-ed45-4bf7-927a-45f25f289d5a admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:28.477095 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a99c46-ece2-4c28-b4e4-9cefbd35d7fc admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:28.481892 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cc5037-d00e-47de-8ce0-83d6e8be958e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:28.482572 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc5037-d00e-47de-8ce0-83d6e8be958e admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:28.482854 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc5037-d00e-47de-8ce0-83d6e8be958e admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:28.483131 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc5037-d00e-47de-8ce0-83d6e8be958e admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:28.483858 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cc5037-d00e-47de-8ce0-83d6e8be958e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:28.485461 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cc5037-d00e-47de-8ce0-83d6e8be958e admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:28.487373 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7603d5f-ed45-4bf7-927a-45f25f289d5a admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:28.492498 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:28.492919 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6933e95-0521-4535-85d5-ad5f6d9a16b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:28.493609 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6933e95-0521-4535-85d5-ad5f6d9a16b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:28.493926 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6933e95-0521-4535-85d5-ad5f6d9a16b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:28.494203 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6933e95-0521-4535-85d5-ad5f6d9a16b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:28.494891 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6933e95-0521-4535-85d5-ad5f6d9a16b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:28.496842 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6933e95-0521-4535-85d5-ad5f6d9a16b1 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:28.499331 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 802/1583] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:40:28.500214 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cc5037-d00e-47de-8ce0-83d6e8be958e admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:28.510618 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:28.511879 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cc5037-d00e-47de-8ce0-83d6e8be958e admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:28.516943 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db3cad8-815a-4e85-a993-8dda4f912555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:28.517628 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db3cad8-815a-4e85-a993-8dda4f912555 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:28.517944 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db3cad8-815a-4e85-a993-8dda4f912555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:28.518239 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db3cad8-815a-4e85-a993-8dda4f912555 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:28.519029 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db3cad8-815a-4e85-a993-8dda4f912555 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:28.521012 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db3cad8-815a-4e85-a993-8dda4f912555 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:28.531531 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 782/1584] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:40:28 2022] POST /identity/v3/projects => generated 403 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:40:28.532456 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6933e95-0521-4535-85d5-ad5f6d9a16b1 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:28.548002 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 803/1585] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:40:28 2022] POST /identity/v3/users => generated 288 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:40:28.548002 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db3cad8-815a-4e85-a993-8dda4f912555 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:28.556379 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db3cad8-815a-4e85-a993-8dda4f912555 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:28.608364 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6933e95-0521-4535-85d5-ad5f6d9a16b1 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:28.613893 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e09b182-8fee-4334-8a59-23376e439bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:28.614969 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09b182-8fee-4334-8a59-23376e439bc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:28.615480 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09b182-8fee-4334-8a59-23376e439bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:28.617505 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09b182-8fee-4334-8a59-23376e439bc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:28.618682 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e09b182-8fee-4334-8a59-23376e439bc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:28.619326 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b3c314-0272-4367-a516-23b70d21e873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:28.620014 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3c314-0272-4367-a516-23b70d21e873 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:28.620306 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3c314-0272-4367-a516-23b70d21e873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:28.620573 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3c314-0272-4367-a516-23b70d21e873 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:28.621117 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e09b182-8fee-4334-8a59-23376e439bc6 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:28.621453 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b3c314-0272-4367-a516-23b70d21e873 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:28.623505 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b3c314-0272-4367-a516-23b70d21e873 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:28.633014 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 783/1586] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:40:28.633904 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e09b182-8fee-4334-8a59-23376e439bc6 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:28.644621 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e09b182-8fee-4334-8a59-23376e439bc6 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:28.651003 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a795122-aa21-4e8a-a93b-ce30c386ed8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:28.651713 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a795122-aa21-4e8a-a93b-ce30c386ed8d admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:28.651977 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a795122-aa21-4e8a-a93b-ce30c386ed8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:28.652267 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a795122-aa21-4e8a-a93b-ce30c386ed8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:28.652977 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a795122-aa21-4e8a-a93b-ce30c386ed8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:28.654617 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a795122-aa21-4e8a-a93b-ce30c386ed8d admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:28.659915 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 804/1587] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:40:28 2022] POST /identity/v3/projects => generated 403 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:40:28.661276 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b3c314-0272-4367-a516-23b70d21e873 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:28.669679 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 784/1588] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:40:28.669679 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a795122-aa21-4e8a-a93b-ce30c386ed8d admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:28.683684 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a795122-aa21-4e8a-a93b-ce30c386ed8d admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:28.728770 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b3c314-0272-4367-a516-23b70d21e873 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:28.734252 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047f6bd0-a504-4372-962d-40124898050e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:28.734960 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047f6bd0-a504-4372-962d-40124898050e admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:28.735243 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047f6bd0-a504-4372-962d-40124898050e admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:28.735515 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047f6bd0-a504-4372-962d-40124898050e admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:28.736245 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047f6bd0-a504-4372-962d-40124898050e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:28.737937 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047f6bd0-a504-4372-962d-40124898050e admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:28.743131 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7504e402-45a0-49df-acfe-9a0cbd3b2322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:28.743888 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7504e402-45a0-49df-acfe-9a0cbd3b2322 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:28.744275 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7504e402-45a0-49df-acfe-9a0cbd3b2322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:28.744559 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7504e402-45a0-49df-acfe-9a0cbd3b2322 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:28.745290 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7504e402-45a0-49df-acfe-9a0cbd3b2322 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:28.746919 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7504e402-45a0-49df-acfe-9a0cbd3b2322 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:28.750117 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 805/1589] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:40:28.750914 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047f6bd0-a504-4372-962d-40124898050e admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:28.763820 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 785/1590] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:40:28.764020 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047f6bd0-a504-4372-962d-40124898050e admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:28.764145 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7504e402-45a0-49df-acfe-9a0cbd3b2322 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:28.766192 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246b2d07-f61d-4340-84ec-9e703cf8f466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:28.767089 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b2d07-f61d-4340-84ec-9e703cf8f466 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:28.767375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b2d07-f61d-4340-84ec-9e703cf8f466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:28.767652 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b2d07-f61d-4340-84ec-9e703cf8f466 admin admin] PATH_INFO: `/v3/projects/2988ac4c59124e02bbcc7049e69dd599/users/71cbad05d1a54b4b9959e2ebbcb720fd/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:28.771678 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7504e402-45a0-49df-acfe-9a0cbd3b2322 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:28.776307 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14267a2-119d-46b9-8b92-567394cfdf58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:28.776968 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14267a2-119d-46b9-8b92-567394cfdf58 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:28.777252 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14267a2-119d-46b9-8b92-567394cfdf58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:28.777525 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14267a2-119d-46b9-8b92-567394cfdf58 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:28.778239 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14267a2-119d-46b9-8b92-567394cfdf58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:28.780291 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14267a2-119d-46b9-8b92-567394cfdf58 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:28.783463 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246b2d07-f61d-4340-84ec-9e703cf8f466 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2988ac4c59124e02bbcc7049e69dd599, user_id=71cbad05d1a54b4b9959e2ebbcb720fd, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:28.784501 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: WARNING py.warnings [None req-246b2d07-f61d-4340-84ec-9e703cf8f466 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 08:40:28.784501 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: warnings.warn(msg) Jun 24 08:40:28.784501 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: Jun 24 08:40:28.786457 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246b2d07-f61d-4340-84ec-9e703cf8f466 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:28.801791 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 786/1591] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:40:28 2022] POST /identity/v3/users => generated 302 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:40:28.802401 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 806/1592] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:40:28 2022] PUT /identity/v3/projects/2988ac4c59124e02bbcc7049e69dd599/users/71cbad05d1a54b4b9959e2ebbcb720fd/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:40:28.803787 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246b2d07-f61d-4340-84ec-9e703cf8f466 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:28.803990 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14267a2-119d-46b9-8b92-567394cfdf58 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:28.815579 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14267a2-119d-46b9-8b92-567394cfdf58 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:28.847002 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:28.868347 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:28.869666 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246b2d07-f61d-4340-84ec-9e703cf8f466 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:28.874758 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86b8e6d-9dd7-4eed-85ef-b683b079912c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:28.875432 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b8e6d-9dd7-4eed-85ef-b683b079912c admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:28.875743 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b8e6d-9dd7-4eed-85ef-b683b079912c admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:28.876023 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b8e6d-9dd7-4eed-85ef-b683b079912c admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:28.876758 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86b8e6d-9dd7-4eed-85ef-b683b079912c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:28.876911 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296438c7-0639-468c-a085-cb0734f1a8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:28.877592 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296438c7-0639-468c-a085-cb0734f1a8bd admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:28.877915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296438c7-0639-468c-a085-cb0734f1a8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:28.878199 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296438c7-0639-468c-a085-cb0734f1a8bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:28.878458 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86b8e6d-9dd7-4eed-85ef-b683b079912c admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:28.878934 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296438c7-0639-468c-a085-cb0734f1a8bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:28.880538 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296438c7-0639-468c-a085-cb0734f1a8bd admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:28.892748 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 807/1593] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:40:28.892748 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86b8e6d-9dd7-4eed-85ef-b683b079912c admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:28.893031 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 787/1594] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:40:28.893031 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296438c7-0639-468c-a085-cb0734f1a8bd admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:28.900990 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86b8e6d-9dd7-4eed-85ef-b683b079912c admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:28.902475 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296438c7-0639-468c-a085-cb0734f1a8bd admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:28.905646 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d723550-5830-4ddc-9d15-45782ed14217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:28.906474 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d723550-5830-4ddc-9d15-45782ed14217 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:28.906766 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d723550-5830-4ddc-9d15-45782ed14217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:28.907048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d723550-5830-4ddc-9d15-45782ed14217 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:28.907170 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd49520-d392-42ca-803a-0795784bdedf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:28.907701 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd49520-d392-42ca-803a-0795784bdedf admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:28.907806 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d723550-5830-4ddc-9d15-45782ed14217 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:28.907992 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd49520-d392-42ca-803a-0795784bdedf admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:28.908270 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd49520-d392-42ca-803a-0795784bdedf admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:28.908990 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd49520-d392-42ca-803a-0795784bdedf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:28.909649 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d723550-5830-4ddc-9d15-45782ed14217 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:28.910607 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd49520-d392-42ca-803a-0795784bdedf admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:28.922235 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 788/1595] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:40:28.922235 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd49520-d392-42ca-803a-0795784bdedf admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:28.933949 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd49520-d392-42ca-803a-0795784bdedf admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:28.935345 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 808/1596] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:40:28 2022] POST /identity/v3/users => generated 308 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:40:28.936132 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d723550-5830-4ddc-9d15-45782ed14217 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:28.941477 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb02606c-8e5a-4e9c-8166-dbf158e5427c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:28.942398 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb02606c-8e5a-4e9c-8166-dbf158e5427c admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:28.942681 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb02606c-8e5a-4e9c-8166-dbf158e5427c admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:28.942955 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb02606c-8e5a-4e9c-8166-dbf158e5427c admin admin] PATH_INFO: `/v3/projects/4de333e57ed34492bee6491335ff4356/users/75a612089b2b46a6b3ff5a6cd9e59579/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:28.946881 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d723550-5830-4ddc-9d15-45782ed14217 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:28.951676 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d5a671-517b-413b-8b35-5e340246e064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:28.952348 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d5a671-517b-413b-8b35-5e340246e064 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:28.952630 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d5a671-517b-413b-8b35-5e340246e064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:28.952905 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d5a671-517b-413b-8b35-5e340246e064 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:28.953663 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d5a671-517b-413b-8b35-5e340246e064 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:28.955309 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d5a671-517b-413b-8b35-5e340246e064 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:28.958414 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb02606c-8e5a-4e9c-8166-dbf158e5427c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4de333e57ed34492bee6491335ff4356, user_id=75a612089b2b46a6b3ff5a6cd9e59579, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:28.961547 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb02606c-8e5a-4e9c-8166-dbf158e5427c admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:28.966642 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 809/1597] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:40:28.967299 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d5a671-517b-413b-8b35-5e340246e064 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:28.977980 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d5a671-517b-413b-8b35-5e340246e064 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:28.982889 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c9cf1b-7c10-4b49-bed2-7d4bbcccd1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:28.983563 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c9cf1b-7c10-4b49-bed2-7d4bbcccd1d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:28.983845 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c9cf1b-7c10-4b49-bed2-7d4bbcccd1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:28.984143 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c9cf1b-7c10-4b49-bed2-7d4bbcccd1d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:28.984873 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c9cf1b-7c10-4b49-bed2-7d4bbcccd1d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:28.986623 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c9cf1b-7c10-4b49-bed2-7d4bbcccd1d2 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:28.988255 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 789/1598] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:40:28 2022] PUT /identity/v3/projects/4de333e57ed34492bee6491335ff4356/users/75a612089b2b46a6b3ff5a6cd9e59579/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:40:28.989329 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d952733e-0732-4e52-9837-fc7770945d55 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:28.989628 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d952733e-0732-4e52-9837-fc7770945d55 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:28.989926 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d952733e-0732-4e52-9837-fc7770945d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:29.007446 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 810/1599] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:40:29.007446 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c9cf1b-7c10-4b49-bed2-7d4bbcccd1d2 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:29.015333 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-d952733e-0732-4e52-9837-fc7770945d55 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:29.017394 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c9cf1b-7c10-4b49-bed2-7d4bbcccd1d2 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:29.017524 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d952733e-0732-4e52-9837-fc7770945d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:29.081169 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc642224-2457-484a-ab4e-63d177b42b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:29.082096 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc642224-2457-484a-ab4e-63d177b42b63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:29.082390 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc642224-2457-484a-ab4e-63d177b42b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:29.082666 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc642224-2457-484a-ab4e-63d177b42b63 admin admin] PATH_INFO: `/v3/projects/96a995413a5146d88dd2d817961a9310/users/0153773c94dd40c49dd740f27a821dfe/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:29.089323 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 790/1600] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:29.090344 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d952733e-0732-4e52-9837-fc7770945d55 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:29.100029 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d952733e-0732-4e52-9837-fc7770945d55 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:29.106045 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc642224-2457-484a-ab4e-63d177b42b63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96a995413a5146d88dd2d817961a9310, user_id=0153773c94dd40c49dd740f27a821dfe, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:29.106951 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db26966-3c13-4a04-99bb-4825b478d80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:29.112791 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db26966-3c13-4a04-99bb-4825b478d80e admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:29.112791 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db26966-3c13-4a04-99bb-4825b478d80e admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:29.112791 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db26966-3c13-4a04-99bb-4825b478d80e admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:29.113160 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc642224-2457-484a-ab4e-63d177b42b63 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:29.113258 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db26966-3c13-4a04-99bb-4825b478d80e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:29.113258 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db26966-3c13-4a04-99bb-4825b478d80e admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:29.121877 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 791/1601] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:40:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:40:29.122960 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db26966-3c13-4a04-99bb-4825b478d80e admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:29.127650 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 811/1602] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:40:29 2022] PUT /identity/v3/projects/96a995413a5146d88dd2d817961a9310/users/0153773c94dd40c49dd740f27a821dfe/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:40:29.128538 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc642224-2457-484a-ab4e-63d177b42b63 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:29.134088 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db26966-3c13-4a04-99bb-4825b478d80e admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:29.192580 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc642224-2457-484a-ab4e-63d177b42b63 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:29.193125 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1ee158-250e-4caf-a033-13f4c10ec787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:29.194692 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ee158-250e-4caf-a033-13f4c10ec787 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:29.194987 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ee158-250e-4caf-a033-13f4c10ec787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:29.195269 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ee158-250e-4caf-a033-13f4c10ec787 admin admin] PATH_INFO: `/v3/projects/147d9b38796148cb996e8c61d1586b61/users/c8215c41d0c54c62b4c9fea630481000/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:29.197628 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a40e2-43c5-42e4-b051-5df00b42f825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:29.198334 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a40e2-43c5-42e4-b051-5df00b42f825 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:29.198621 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a40e2-43c5-42e4-b051-5df00b42f825 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:29.198912 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a40e2-43c5-42e4-b051-5df00b42f825 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:29.199673 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170a40e2-43c5-42e4-b051-5df00b42f825 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:29.201274 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170a40e2-43c5-42e4-b051-5df00b42f825 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:29.212074 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1ee158-250e-4caf-a033-13f4c10ec787 admin admin] RBAC: Authorizing `identity:create_grant(project_id=147d9b38796148cb996e8c61d1586b61, user_id=c8215c41d0c54c62b4c9fea630481000, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:29.212646 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 812/1603] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:40:29 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:40:29.213470 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a40e2-43c5-42e4-b051-5df00b42f825 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:29.214801 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1ee158-250e-4caf-a033-13f4c10ec787 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:29.224542 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a40e2-43c5-42e4-b051-5df00b42f825 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:29.230374 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 792/1604] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:40:29 2022] PUT /identity/v3/projects/147d9b38796148cb996e8c61d1586b61/users/c8215c41d0c54c62b4c9fea630481000/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:40:29.235360 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbd65ee-704e-4edf-a136-c61206d6fe1e None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:29.235653 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbd65ee-704e-4edf-a136-c61206d6fe1e None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:29.235986 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbd65ee-704e-4edf-a136-c61206d6fe1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:29.237142 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:29.254934 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-5dbd65ee-704e-4edf-a136-c61206d6fe1e None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:29.256161 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-5dbd65ee-704e-4edf-a136-c61206d6fe1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:29.283177 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:29.286814 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843da179-ac5a-4cda-b0e8-65ca054760e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:29.288420 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843da179-ac5a-4cda-b0e8-65ca054760e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:29.288704 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843da179-ac5a-4cda-b0e8-65ca054760e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:29.289013 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843da179-ac5a-4cda-b0e8-65ca054760e3 admin admin] PATH_INFO: `/v3/projects/dfa535df5dd14d69bb3ca125cbd9fe13/users/71e0fd73e5c9430c813af1de9ccd4a52/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:29.309330 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843da179-ac5a-4cda-b0e8-65ca054760e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfa535df5dd14d69bb3ca125cbd9fe13, user_id=71e0fd73e5c9430c813af1de9ccd4a52, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:29.312093 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843da179-ac5a-4cda-b0e8-65ca054760e3 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:29.321850 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 793/1605] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:29.323201 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbd65ee-704e-4edf-a136-c61206d6fe1e None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:29.330022 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 813/1606] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:40:29 2022] PUT /identity/v3/projects/dfa535df5dd14d69bb3ca125cbd9fe13/users/71e0fd73e5c9430c813af1de9ccd4a52/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:40:29.330022 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843da179-ac5a-4cda-b0e8-65ca054760e3 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:29.333960 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbd65ee-704e-4edf-a136-c61206d6fe1e None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:29.392356 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843da179-ac5a-4cda-b0e8-65ca054760e3 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:29.395773 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b8dd6-1ca7-48b8-a872-f2695edb6567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:29.396680 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b8dd6-1ca7-48b8-a872-f2695edb6567 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:29.397418 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628e040e-ae31-4b7f-bd5c-cce067c1eecc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:29.397969 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b8dd6-1ca7-48b8-a872-f2695edb6567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:29.398350 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e040e-ae31-4b7f-bd5c-cce067c1eecc service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:29.398585 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b8dd6-1ca7-48b8-a872-f2695edb6567 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:29.398818 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e040e-ae31-4b7f-bd5c-cce067c1eecc service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:29.399190 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e040e-ae31-4b7f-bd5c-cce067c1eecc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:29.399776 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156b8dd6-1ca7-48b8-a872-f2695edb6567 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:29.401540 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156b8dd6-1ca7-48b8-a872-f2695edb6567 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:29.412004 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628e040e-ae31-4b7f-bd5c-cce067c1eecc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:29.413097 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 794/1607] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:40:29 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:40:29.413635 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628e040e-ae31-4b7f-bd5c-cce067c1eecc service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:29.413923 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b8dd6-1ca7-48b8-a872-f2695edb6567 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:29.423977 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b8dd6-1ca7-48b8-a872-f2695edb6567 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:29.424182 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 814/1608] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:40:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:40:29.428938 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075f29bd-e81d-43af-8bd5-de5377a36735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:29.429857 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f29bd-e81d-43af-8bd5-de5377a36735 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:29.430151 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f29bd-e81d-43af-8bd5-de5377a36735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:29.430434 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f29bd-e81d-43af-8bd5-de5377a36735 admin admin] PATH_INFO: `/v3/projects/c859b23c7e2d4d5bae1d73ec8f1b32c4/users/6da677ca99d34f46a9a067e0d35a8ed4/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:29.430738 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf5e2c-b8c8-4bc1-8eb8-c0debb7b4b47 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:29.431158 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf5e2c-b8c8-4bc1-8eb8-c0debb7b4b47 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:29.431545 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf5e2c-b8c8-4bc1-8eb8-c0debb7b4b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:29.447671 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075f29bd-e81d-43af-8bd5-de5377a36735 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c859b23c7e2d4d5bae1d73ec8f1b32c4, user_id=6da677ca99d34f46a9a067e0d35a8ed4, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:29.450537 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075f29bd-e81d-43af-8bd5-de5377a36735 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:29.461896 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-32cf5e2c-b8c8-4bc1-8eb8-c0debb7b4b47 None None] MFA Rules not processed for user `0153773c94dd40c49dd740f27a821dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:29.463145 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-32cf5e2c-b8c8-4bc1-8eb8-c0debb7b4b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:29.467343 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 795/1609] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:40:29 2022] PUT /identity/v3/projects/c859b23c7e2d4d5bae1d73ec8f1b32c4/users/6da677ca99d34f46a9a067e0d35a8ed4/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:40:29.467920 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075f29bd-e81d-43af-8bd5-de5377a36735 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:29.544549 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075f29bd-e81d-43af-8bd5-de5377a36735 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:29.549812 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470943d2-bd9c-41ab-a46e-facf3f0f8b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:29.550774 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470943d2-bd9c-41ab-a46e-facf3f0f8b0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:29.551065 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470943d2-bd9c-41ab-a46e-facf3f0f8b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:29.551346 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470943d2-bd9c-41ab-a46e-facf3f0f8b0d admin admin] PATH_INFO: `/v3/projects/d1a4e12b8f724e27ad10bf67d10bbb1e/users/9dcbcace6e804c719903ae91c09d6c42/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:29.558904 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 815/1610] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:29.558904 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc788c05-26f2-4b87-a8cf-e9e2d4984ee0 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:29.558904 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc788c05-26f2-4b87-a8cf-e9e2d4984ee0 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:29.558904 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc788c05-26f2-4b87-a8cf-e9e2d4984ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:29.576676 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-cc788c05-26f2-4b87-a8cf-e9e2d4984ee0 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:29.577935 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-cc788c05-26f2-4b87-a8cf-e9e2d4984ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:29.587166 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 816/1611] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:29.591851 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470943d2-bd9c-41ab-a46e-facf3f0f8b0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1a4e12b8f724e27ad10bf67d10bbb1e, user_id=9dcbcace6e804c719903ae91c09d6c42, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:29.595528 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470943d2-bd9c-41ab-a46e-facf3f0f8b0d admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:29.598945 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f293961-dc9d-4769-88c7-fcedc87f4d08 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:29.599298 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f293961-dc9d-4769-88c7-fcedc87f4d08 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:29.599530 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f293961-dc9d-4769-88c7-fcedc87f4d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:29.619758 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-8f293961-dc9d-4769-88c7-fcedc87f4d08 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:29.624166 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-8f293961-dc9d-4769-88c7-fcedc87f4d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:29.635356 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 796/1612] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:40:29 2022] PUT /identity/v3/projects/d1a4e12b8f724e27ad10bf67d10bbb1e/users/9dcbcace6e804c719903ae91c09d6c42/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:40:29.635356 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470943d2-bd9c-41ab-a46e-facf3f0f8b0d admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:29.654255 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:29.674097 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:29.706261 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470943d2-bd9c-41ab-a46e-facf3f0f8b0d admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:29.711847 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c73405e-73b4-4316-8683-4a939edb94e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:29.712806 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73405e-73b4-4316-8683-4a939edb94e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:29.713294 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73405e-73b4-4316-8683-4a939edb94e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:29.713667 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73405e-73b4-4316-8683-4a939edb94e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:29.716855 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 817/1613] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:29.718109 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f293961-dc9d-4769-88c7-fcedc87f4d08 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:29.726361 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c73405e-73b4-4316-8683-4a939edb94e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:29.727971 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c73405e-73b4-4316-8683-4a939edb94e0 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:29.730520 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f293961-dc9d-4769-88c7-fcedc87f4d08 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:29.738004 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ed9fc4-2d61-4186-9c34-ef9bfad1baf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:29.739155 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed9fc4-2d61-4186-9c34-ef9bfad1baf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:29.739366 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 797/1614] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:40:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:40:29.739555 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed9fc4-2d61-4186-9c34-ef9bfad1baf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:29.741050 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed9fc4-2d61-4186-9c34-ef9bfad1baf9 admin admin] PATH_INFO: `/v3/projects/b362cc2cf2484182afb8365a5fe5bd98/users/b8001fbcde0244089c3e6f999b339464/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:29.741664 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e37085-a84e-4183-a085-913055d07940 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:29.741866 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e37085-a84e-4183-a085-913055d07940 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:29.742237 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e37085-a84e-4183-a085-913055d07940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:29.763710 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ed9fc4-2d61-4186-9c34-ef9bfad1baf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b362cc2cf2484182afb8365a5fe5bd98, user_id=b8001fbcde0244089c3e6f999b339464, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:29.767301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ed9fc4-2d61-4186-9c34-ef9bfad1baf9 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:29.781766 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-67e37085-a84e-4183-a085-913055d07940 None None] MFA Rules not processed for user `6da677ca99d34f46a9a067e0d35a8ed4`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:29.783658 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-67e37085-a84e-4183-a085-913055d07940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:29.790323 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 818/1615] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:40:29 2022] PUT /identity/v3/projects/b362cc2cf2484182afb8365a5fe5bd98/users/b8001fbcde0244089c3e6f999b339464/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:40:29.794058 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b314b584-daf0-48f8-bce6-d9a9f8385a0e None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:29.794058 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b314b584-daf0-48f8-bce6-d9a9f8385a0e None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:29.794058 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b314b584-daf0-48f8-bce6-d9a9f8385a0e None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:29.794923 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 819/1616] 10.222.0.13 () {54 vars in 961 bytes} [Fri Jun 24 08:40:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:40:29.795754 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b314b584-daf0-48f8-bce6-d9a9f8385a0e None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:29.798023 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b314b584-daf0-48f8-bce6-d9a9f8385a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:29.876308 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 798/1617] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:29.877513 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e37085-a84e-4183-a085-913055d07940 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:29.907026 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b314b584-daf0-48f8-bce6-d9a9f8385a0e None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:29.912920 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9a1f9b-ef50-4255-adc0-7bb6f5e7c5aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:29.915105 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a1f9b-ef50-4255-adc0-7bb6f5e7c5aa service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:29.915605 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a1f9b-ef50-4255-adc0-7bb6f5e7c5aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:29.916098 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a1f9b-ef50-4255-adc0-7bb6f5e7c5aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:29.986575 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e37085-a84e-4183-a085-913055d07940 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:30.001063 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0db2c5b-7343-4e98-bbd5-04b3bf92673b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:30.007422 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9a1f9b-ef50-4255-adc0-7bb6f5e7c5aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:30.008364 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db2c5b-7343-4e98-bbd5-04b3bf92673b admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:30.008737 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db2c5b-7343-4e98-bbd5-04b3bf92673b admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:30.009342 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db2c5b-7343-4e98-bbd5-04b3bf92673b admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:30.009462 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9a1f9b-ef50-4255-adc0-7bb6f5e7c5aa service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:30.010254 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0db2c5b-7343-4e98-bbd5-04b3bf92673b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:30.012318 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0db2c5b-7343-4e98-bbd5-04b3bf92673b admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:30.027881 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 820/1618] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:40:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:40:30.028558 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9a1f9b-ef50-4255-adc0-7bb6f5e7c5aa service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:30.042030 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 799/1619] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:40:29 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:40:30.043234 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6efe39-c0c6-4e79-a2c6-2119580e0f02 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:30.043234 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6efe39-c0c6-4e79-a2c6-2119580e0f02 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:30.044329 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6efe39-c0c6-4e79-a2c6-2119580e0f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:30.049570 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9a1f9b-ef50-4255-adc0-7bb6f5e7c5aa service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:30.060216 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f535cd9e-505d-46a9-b488-902a23f8f4fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:30.062950 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535cd9e-505d-46a9-b488-902a23f8f4fd service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:30.063458 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535cd9e-505d-46a9-b488-902a23f8f4fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:30.064853 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535cd9e-505d-46a9-b488-902a23f8f4fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:30.085741 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f535cd9e-505d-46a9-b488-902a23f8f4fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:30.088768 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f535cd9e-505d-46a9-b488-902a23f8f4fd service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:30.092891 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-ba6efe39-c0c6-4e79-a2c6-2119580e0f02 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:30.099823 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-ba6efe39-c0c6-4e79-a2c6-2119580e0f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:30.105622 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 821/1620] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:40:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:40:30.107685 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61e75d-7375-4fda-be09-fe7912c6e662 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:30.108346 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61e75d-7375-4fda-be09-fe7912c6e662 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:30.109656 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61e75d-7375-4fda-be09-fe7912c6e662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:30.123393 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 800/1621] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:30.124662 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d97972-ede2-462d-b203-2a6468ced5ed None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:30.125011 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d97972-ede2-462d-b203-2a6468ced5ed None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:30.125296 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d97972-ede2-462d-b203-2a6468ced5ed None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:30.128612 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 801/1622] 10.222.0.13 () {54 vars in 961 bytes} [Fri Jun 24 08:40:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:40:30.129532 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d97972-ede2-462d-b203-2a6468ced5ed None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:30.134347 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-fa61e75d-7375-4fda-be09-fe7912c6e662 None None] MFA Rules not processed for user `74bb1bc6821b4500b26f160ccae5b7d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:30.136231 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-fa61e75d-7375-4fda-be09-fe7912c6e662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:30.150931 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d97972-ede2-462d-b203-2a6468ced5ed None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:30.173885 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d50571d-4298-4cd5-88db-69da65c728ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:30.176526 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50571d-4298-4cd5-88db-69da65c728ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:30.177254 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50571d-4298-4cd5-88db-69da65c728ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:30.178262 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50571d-4298-4cd5-88db-69da65c728ae admin admin] PATH_INFO: `/v3/projects/d1a4e12b8f724e27ad10bf67d10bbb1e/users/9dcbcace6e804c719903ae91c09d6c42/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:30.183455 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d50571d-4298-4cd5-88db-69da65c728ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1a4e12b8f724e27ad10bf67d10bbb1e, user_id=9dcbcace6e804c719903ae91c09d6c42, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:30.186651 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d50571d-4298-4cd5-88db-69da65c728ae admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:30.196394 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:30.213784 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 802/1623] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:40:30 2022] PUT /identity/v3/projects/d1a4e12b8f724e27ad10bf67d10bbb1e/users/9dcbcace6e804c719903ae91c09d6c42/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:40:30.213784 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89881cb-9959-4875-8259-8bec12998f8b None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:30.213784 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89881cb-9959-4875-8259-8bec12998f8b None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:30.213784 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89881cb-9959-4875-8259-8bec12998f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:30.252708 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-c89881cb-9959-4875-8259-8bec12998f8b None None] MFA Rules not processed for user `74bb1bc6821b4500b26f160ccae5b7d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:30.254336 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-c89881cb-9959-4875-8259-8bec12998f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:30.278788 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 822/1624] 10.222.0.13 () {58 vars in 1058 bytes} [Fri Jun 24 08:40:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:30.280776 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61e75d-7375-4fda-be09-fe7912c6e662 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:30.293876 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61e75d-7375-4fda-be09-fe7912c6e662 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:30.304807 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:30.427145 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452df99c-b9bb-4ea2-a159-30dc044beee0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:30.428072 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452df99c-b9bb-4ea2-a159-30dc044beee0 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:30.428434 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452df99c-b9bb-4ea2-a159-30dc044beee0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:30.428833 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452df99c-b9bb-4ea2-a159-30dc044beee0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:30.434672 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 803/1625] 10.222.0.13 () {58 vars in 1058 bytes} [Fri Jun 24 08:40:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:30.438656 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1dc2d-52df-4845-8d10-b7ed028b9807 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:30.440573 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1dc2d-52df-4845-8d10-b7ed028b9807 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:30.440717 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1dc2d-52df-4845-8d10-b7ed028b9807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:30.514250 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-39d1dc2d-52df-4845-8d10-b7ed028b9807 None None] MFA Rules not processed for user `9dcbcace6e804c719903ae91c09d6c42`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:30.518446 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-39d1dc2d-52df-4845-8d10-b7ed028b9807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:30.577126 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452df99c-b9bb-4ea2-a159-30dc044beee0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:30.579881 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452df99c-b9bb-4ea2-a159-30dc044beee0 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:30.594678 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 823/1626] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:40:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:40:30.595825 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452df99c-b9bb-4ea2-a159-30dc044beee0 service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:30.612063 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452df99c-b9bb-4ea2-a159-30dc044beee0 service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:30.621801 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a767a17-9ab4-4069-9ae9-85ab05bcc361 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74bb1bc6821b4500b26f160ccae5b7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:30.623108 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a767a17-9ab4-4069-9ae9-85ab05bcc361 service nova] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:30.623492 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a767a17-9ab4-4069-9ae9-85ab05bcc361 service nova] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:30.624546 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a767a17-9ab4-4069-9ae9-85ab05bcc361 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:30.761950 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a767a17-9ab4-4069-9ae9-85ab05bcc361 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:30.765751 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 804/1627] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:30 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:30.765940 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a767a17-9ab4-4069-9ae9-85ab05bcc361 service nova] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:30.766051 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d1dc2d-52df-4845-8d10-b7ed028b9807 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:30.791012 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d1dc2d-52df-4845-8d10-b7ed028b9807 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:30.793371 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 824/1628] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:40:30 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:40:30.795444 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a767a17-9ab4-4069-9ae9-85ab05bcc361 service nova] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:30.809436 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fffbe6-8e7f-4590-b399-faa8d943b83d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74bb1bc6821b4500b26f160ccae5b7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:30.810255 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fffbe6-8e7f-4590-b399-faa8d943b83d service nova] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:30.810578 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fffbe6-8e7f-4590-b399-faa8d943b83d service nova] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:30.810887 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fffbe6-8e7f-4590-b399-faa8d943b83d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:30.851637 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:30.948871 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a767a17-9ab4-4069-9ae9-85ab05bcc361 service nova] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:30.950941 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fffbe6-8e7f-4590-b399-faa8d943b83d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:30.961437 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed5da9-5b1b-40ee-8d02-fdc86db4f6df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:30.965457 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed5da9-5b1b-40ee-8d02-fdc86db4f6df service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:30.965830 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed5da9-5b1b-40ee-8d02-fdc86db4f6df service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:30.965958 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fffbe6-8e7f-4590-b399-faa8d943b83d service nova] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:30.966321 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed5da9-5b1b-40ee-8d02-fdc86db4f6df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:30.994171 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ed5da9-5b1b-40ee-8d02-fdc86db4f6df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:30.995773 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ed5da9-5b1b-40ee-8d02-fdc86db4f6df service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:30.996277 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 805/1629] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:40:30 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:40:31.039626 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 825/1630] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:40:30 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:40:31.404485 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a60a8-9cf0-42a3-ba82-64ce0ba6d096 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:31.405036 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a60a8-9cf0-42a3-ba82-64ce0ba6d096 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:31.405036 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a60a8-9cf0-42a3-ba82-64ce0ba6d096 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:31.406973 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 806/1631] 10.222.0.13 () {54 vars in 912 bytes} [Fri Jun 24 08:40:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:40:31.408657 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed5da9-5b1b-40ee-8d02-fdc86db4f6df service cinder] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:31.413466 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ad0af-1e8c-40d4-938d-6bd12d65660c None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:31.413844 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ad0af-1e8c-40d4-938d-6bd12d65660c None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:31.414067 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ad0af-1e8c-40d4-938d-6bd12d65660c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:31.435473 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed5da9-5b1b-40ee-8d02-fdc86db4f6df service cinder] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:31.456917 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c35ad2-167c-47db-a172-60ae7a079700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:31.458960 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c35ad2-167c-47db-a172-60ae7a079700 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:31.460514 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c35ad2-167c-47db-a172-60ae7a079700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:31.461043 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c35ad2-167c-47db-a172-60ae7a079700 admin admin] PATH_INFO: `/v3/users/2be4a70caeb4445a90008b8213d6f8bf` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:31.462938 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c35ad2-167c-47db-a172-60ae7a079700 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2be4a70caeb4445a90008b8213d6f8bf)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:31.463915 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: WARNING py.warnings [None req-54c35ad2-167c-47db-a172-60ae7a079700 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 08:40:31.463915 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: warnings.warn(msg) Jun 24 08:40:31.463915 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: Jun 24 08:40:31.468051 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c35ad2-167c-47db-a172-60ae7a079700 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:31.475298 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-fd4ad0af-1e8c-40d4-938d-6bd12d65660c None None] MFA Rules not processed for user `74bb1bc6821b4500b26f160ccae5b7d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:31.476650 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-fd4ad0af-1e8c-40d4-938d-6bd12d65660c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:31.511702 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 807/1632] 10.222.0.13 () {58 vars in 1009 bytes} [Fri Jun 24 08:40:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:40:31.527267 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4ad0af-1e8c-40d4-938d-6bd12d65660c None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:31.539246 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4ad0af-1e8c-40d4-938d-6bd12d65660c None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:31.547484 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb194ae-3cb8-4f44-9ef5-f799f5ff25cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:31.548531 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb194ae-3cb8-4f44-9ef5-f799f5ff25cb service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:31.548728 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb194ae-3cb8-4f44-9ef5-f799f5ff25cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:31.549146 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb194ae-3cb8-4f44-9ef5-f799f5ff25cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:31.571077 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb194ae-3cb8-4f44-9ef5-f799f5ff25cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:31.573018 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb194ae-3cb8-4f44-9ef5-f799f5ff25cb service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:31.589501 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 808/1633] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:40:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:40:31.591623 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6795c-2643-46d4-9590-956ec659ad24 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:31.591855 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6795c-2643-46d4-9590-956ec659ad24 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:31.592141 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6795c-2643-46d4-9590-956ec659ad24 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:31.593764 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 809/1634] 10.222.0.13 () {54 vars in 912 bytes} [Fri Jun 24 08:40:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:40:31.601351 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f40c9d7-2130-4838-9d00-bcd8f8fe5313 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:31.601647 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f40c9d7-2130-4838-9d00-bcd8f8fe5313 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:31.602017 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f40c9d7-2130-4838-9d00-bcd8f8fe5313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:31.602867 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-54c35ad2-167c-47db-a172-60ae7a079700 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2be4a70caeb4445a90008b8213d6f8bf'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:31.626953 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-2f40c9d7-2130-4838-9d00-bcd8f8fe5313 None None] MFA Rules not processed for user `74bb1bc6821b4500b26f160ccae5b7d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:31.628351 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-2f40c9d7-2130-4838-9d00-bcd8f8fe5313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:31.649144 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 810/1635] 10.222.0.13 () {58 vars in 1009 bytes} [Fri Jun 24 08:40:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:40:31.666425 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f40c9d7-2130-4838-9d00-bcd8f8fe5313 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:31.682319 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-54c35ad2-167c-47db-a172-60ae7a079700 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2be4a70caeb4445a90008b8213d6f8bf'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:31.693994 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f40c9d7-2130-4838-9d00-bcd8f8fe5313 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:31.701188 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d276e328-4002-4daa-a05d-45131d760a98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:31.702170 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d276e328-4002-4daa-a05d-45131d760a98 service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:31.702602 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d276e328-4002-4daa-a05d-45131d760a98 service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:31.702974 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d276e328-4002-4daa-a05d-45131d760a98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:31.706274 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-54c35ad2-167c-47db-a172-60ae7a079700 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2be4a70caeb4445a90008b8213d6f8bf'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:31.707262 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-54c35ad2-167c-47db-a172-60ae7a079700 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2be4a70caeb4445a90008b8213d6f8bf'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:31.708483 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-54c35ad2-167c-47db-a172-60ae7a079700 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2be4a70caeb4445a90008b8213d6f8bf'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:31.739867 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d276e328-4002-4daa-a05d-45131d760a98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:31.743202 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d276e328-4002-4daa-a05d-45131d760a98 service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:31.756476 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 826/1636] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:40:31 2022] DELETE /identity/v3/users/2be4a70caeb4445a90008b8213d6f8bf => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:40:31.765641 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951be7e-3117-4575-8a09-043c783cbd61 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:31.766000 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951be7e-3117-4575-8a09-043c783cbd61 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:31.766259 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951be7e-3117-4575-8a09-043c783cbd61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:31.779531 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 811/1637] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:40:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:40:31.812252 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-b951be7e-3117-4575-8a09-043c783cbd61 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:31.813614 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b951be7e-3117-4575-8a09-043c783cbd61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:31.938576 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 827/1638] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:32.039427 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d276e328-4002-4daa-a05d-45131d760a98 service cinder] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:32.057216 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d276e328-4002-4daa-a05d-45131d760a98 service cinder] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:32.065386 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:32.114883 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951be7e-3117-4575-8a09-043c783cbd61 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:32.130657 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b951be7e-3117-4575-8a09-043c783cbd61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:32.153986 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951be7e-3117-4575-8a09-043c783cbd61 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:32.170367 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf419ce4-07f7-4f08-903c-0e93e9e65413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:32.171269 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf419ce4-07f7-4f08-903c-0e93e9e65413 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:32.171555 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf419ce4-07f7-4f08-903c-0e93e9e65413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:32.171836 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf419ce4-07f7-4f08-903c-0e93e9e65413 admin admin] PATH_INFO: `/v3/users/6da677ca99d34f46a9a067e0d35a8ed4` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:32.174916 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf419ce4-07f7-4f08-903c-0e93e9e65413 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6da677ca99d34f46a9a067e0d35a8ed4)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:32.179428 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf419ce4-07f7-4f08-903c-0e93e9e65413 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:32.195529 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b3c5fb-66c8-4776-b857-e5fe50497d6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:32.196200 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3c5fb-66c8-4776-b857-e5fe50497d6b service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:32.196615 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3c5fb-66c8-4776-b857-e5fe50497d6b service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:32.196837 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3c5fb-66c8-4776-b857-e5fe50497d6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:32.222036 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b3c5fb-66c8-4776-b857-e5fe50497d6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:32.226155 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b3c5fb-66c8-4776-b857-e5fe50497d6b service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:32.241117 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 812/1639] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:40:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:40:32.252063 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:32.289015 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-bf419ce4-07f7-4f08-903c-0e93e9e65413 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6da677ca99d34f46a9a067e0d35a8ed4'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:32.314726 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-bf419ce4-07f7-4f08-903c-0e93e9e65413 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6da677ca99d34f46a9a067e0d35a8ed4'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:32.327972 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-bf419ce4-07f7-4f08-903c-0e93e9e65413 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6da677ca99d34f46a9a067e0d35a8ed4'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:32.328814 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-bf419ce4-07f7-4f08-903c-0e93e9e65413 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6da677ca99d34f46a9a067e0d35a8ed4'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:32.329683 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-bf419ce4-07f7-4f08-903c-0e93e9e65413 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6da677ca99d34f46a9a067e0d35a8ed4'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:32.354366 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 828/1640] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:40:32 2022] DELETE /identity/v3/users/6da677ca99d34f46a9a067e0d35a8ed4 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:40:32.356456 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07875616-768e-4939-ba2e-430fdc1f96c4 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:32.356799 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07875616-768e-4939-ba2e-430fdc1f96c4 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:32.357081 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07875616-768e-4939-ba2e-430fdc1f96c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:32.395273 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-07875616-768e-4939-ba2e-430fdc1f96c4 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:32.398378 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-07875616-768e-4939-ba2e-430fdc1f96c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:32.551254 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 813/1641] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:32.569226 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf419ce4-07f7-4f08-903c-0e93e9e65413 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:32.570911 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-bf419ce4-07f7-4f08-903c-0e93e9e65413 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:32.689905 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf419ce4-07f7-4f08-903c-0e93e9e65413 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:32.694857 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737a892c-e906-4610-9ad7-b7eceb424018 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:32.695678 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737a892c-e906-4610-9ad7-b7eceb424018 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:32.696154 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737a892c-e906-4610-9ad7-b7eceb424018 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:32.696424 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737a892c-e906-4610-9ad7-b7eceb424018 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:32.712820 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737a892c-e906-4610-9ad7-b7eceb424018 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:32.714772 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737a892c-e906-4610-9ad7-b7eceb424018 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:32.745946 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 829/1642] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:40:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:40:34.633756 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07875616-768e-4939-ba2e-430fdc1f96c4 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:34.636320 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-07875616-768e-4939-ba2e-430fdc1f96c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:34.660407 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07875616-768e-4939-ba2e-430fdc1f96c4 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:34.679640 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0499c1-454f-4388-8893-85b389c04fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:34.680600 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0499c1-454f-4388-8893-85b389c04fd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:34.680892 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0499c1-454f-4388-8893-85b389c04fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:34.681569 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0499c1-454f-4388-8893-85b389c04fd0 admin admin] PATH_INFO: `/v3/projects/35fe6df7ad774eb8ad6a7ccf008276cf` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:34.683100 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0499c1-454f-4388-8893-85b389c04fd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35fe6df7ad774eb8ad6a7ccf008276cf)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:34.684194 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: WARNING py.warnings [None req-5d0499c1-454f-4388-8893-85b389c04fd0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 08:40:34.684194 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: warnings.warn(msg) Jun 24 08:40:34.684194 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: Jun 24 08:40:34.686113 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0499c1-454f-4388-8893-85b389c04fd0 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:34.808146 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-5d0499c1-454f-4388-8893-85b389c04fd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35fe6df7ad774eb8ad6a7ccf008276cf', 'request_id': 'req-5d0499c1-454f-4388-8893-85b389c04fd0'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:34.838613 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-5d0499c1-454f-4388-8893-85b389c04fd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35fe6df7ad774eb8ad6a7ccf008276cf', 'request_id': 'req-5d0499c1-454f-4388-8893-85b389c04fd0'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:34.866413 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-5d0499c1-454f-4388-8893-85b389c04fd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35fe6df7ad774eb8ad6a7ccf008276cf', 'request_id': 'req-5d0499c1-454f-4388-8893-85b389c04fd0'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:34.910059 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-5d0499c1-454f-4388-8893-85b389c04fd0 admin admin] The token cache is being invalidate because project 35fe6df7ad774eb8ad6a7ccf008276cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:40:34.910059 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-5d0499c1-454f-4388-8893-85b389c04fd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:34.911564 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-5d0499c1-454f-4388-8893-85b389c04fd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:34.919072 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 814/1643] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:40:34 2022] DELETE /identity/v3/projects/35fe6df7ad774eb8ad6a7ccf008276cf => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:40:34.921268 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737a892c-e906-4610-9ad7-b7eceb424018 service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:34.935608 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737a892c-e906-4610-9ad7-b7eceb424018 service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:35.093186 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0499c1-454f-4388-8893-85b389c04fd0 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:35.093186 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-5d0499c1-454f-4388-8893-85b389c04fd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:35.093678 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c88aa6-6708-4215-91e7-4d438e77f4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:35.093678 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c88aa6-6708-4215-91e7-4d438e77f4ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:35.093678 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c88aa6-6708-4215-91e7-4d438e77f4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:35.093678 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c88aa6-6708-4215-91e7-4d438e77f4ad admin admin] PATH_INFO: `/v3/users/9dcbcace6e804c719903ae91c09d6c42` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:35.094584 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c88aa6-6708-4215-91e7-4d438e77f4ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=9dcbcace6e804c719903ae91c09d6c42)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:35.096544 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c88aa6-6708-4215-91e7-4d438e77f4ad admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:35.101193 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0499c1-454f-4388-8893-85b389c04fd0 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:35.120674 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974bc341-8161-4134-b553-a10b43d2053c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:35.138084 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974bc341-8161-4134-b553-a10b43d2053c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:35.141594 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974bc341-8161-4134-b553-a10b43d2053c admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:35.141940 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974bc341-8161-4134-b553-a10b43d2053c admin admin] PATH_INFO: `/v3/projects/c859b23c7e2d4d5bae1d73ec8f1b32c4` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:35.144009 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974bc341-8161-4134-b553-a10b43d2053c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c859b23c7e2d4d5bae1d73ec8f1b32c4)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:35.151349 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974bc341-8161-4134-b553-a10b43d2053c admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:35.247035 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-f0c88aa6-6708-4215-91e7-4d438e77f4ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9dcbcace6e804c719903ae91c09d6c42'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:35.249983 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:35.283315 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:35.286562 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-f0c88aa6-6708-4215-91e7-4d438e77f4ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9dcbcace6e804c719903ae91c09d6c42'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:35.313481 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-f0c88aa6-6708-4215-91e7-4d438e77f4ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9dcbcace6e804c719903ae91c09d6c42'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:35.321296 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-f0c88aa6-6708-4215-91e7-4d438e77f4ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9dcbcace6e804c719903ae91c09d6c42'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:35.322112 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-f0c88aa6-6708-4215-91e7-4d438e77f4ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9dcbcace6e804c719903ae91c09d6c42'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:35.341666 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-974bc341-8161-4134-b553-a10b43d2053c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c859b23c7e2d4d5bae1d73ec8f1b32c4', 'request_id': 'req-974bc341-8161-4134-b553-a10b43d2053c'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:35.364417 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-974bc341-8161-4134-b553-a10b43d2053c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c859b23c7e2d4d5bae1d73ec8f1b32c4', 'request_id': 'req-974bc341-8161-4134-b553-a10b43d2053c'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:35.368443 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 830/1644] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:40:34 2022] DELETE /identity/v3/users/9dcbcace6e804c719903ae91c09d6c42 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:40:35.371797 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8aef5-9ded-42a0-9244-261856d5bf16 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:35.372287 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8aef5-9ded-42a0-9244-261856d5bf16 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:35.374061 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8aef5-9ded-42a0-9244-261856d5bf16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:35.399654 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-974bc341-8161-4134-b553-a10b43d2053c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c859b23c7e2d4d5bae1d73ec8f1b32c4', 'request_id': 'req-974bc341-8161-4134-b553-a10b43d2053c'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:35.423796 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-974bc341-8161-4134-b553-a10b43d2053c admin admin] The token cache is being invalidate because project c859b23c7e2d4d5bae1d73ec8f1b32c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:40:35.424413 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-974bc341-8161-4134-b553-a10b43d2053c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:35.425293 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-974bc341-8161-4134-b553-a10b43d2053c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:35.427511 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 815/1645] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:40:35 2022] DELETE /identity/v3/projects/c859b23c7e2d4d5bae1d73ec8f1b32c4 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:40:35.435089 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6fcc61-2435-4454-9076-2be729ad997f None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:35.435692 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6fcc61-2435-4454-9076-2be729ad997f None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:35.436165 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6fcc61-2435-4454-9076-2be729ad997f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:35.468042 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-b2b8aef5-9ded-42a0-9244-261856d5bf16 None None] MFA Rules not processed for user `b8001fbcde0244089c3e6f999b339464`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:35.470143 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b2b8aef5-9ded-42a0-9244-261856d5bf16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:35.475546 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-0f6fcc61-2435-4454-9076-2be729ad997f None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:35.477788 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-0f6fcc61-2435-4454-9076-2be729ad997f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:35.646340 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 831/1646] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:35 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:35.646583 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 816/1647] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:35.647776 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b8aef5-9ded-42a0-9244-261856d5bf16 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:35.653431 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6fcc61-2435-4454-9076-2be729ad997f None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:35.653963 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b2b8aef5-9ded-42a0-9244-261856d5bf16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:35.655359 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-0f6fcc61-2435-4454-9076-2be729ad997f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:35.679761 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6fcc61-2435-4454-9076-2be729ad997f None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:35.693352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868e1673-2b7d-4a69-8ffc-1c217a2a0e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:35.695439 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e1673-2b7d-4a69-8ffc-1c217a2a0e28 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:35.695805 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e1673-2b7d-4a69-8ffc-1c217a2a0e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:35.696356 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e1673-2b7d-4a69-8ffc-1c217a2a0e28 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:35.697292 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868e1673-2b7d-4a69-8ffc-1c217a2a0e28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:35.699820 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b8aef5-9ded-42a0-9244-261856d5bf16 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:35.701967 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868e1673-2b7d-4a69-8ffc-1c217a2a0e28 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:35.728411 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f3d749-a110-4129-98b9-2fb15202f562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:35.729518 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f3d749-a110-4129-98b9-2fb15202f562 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:35.729988 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f3d749-a110-4129-98b9-2fb15202f562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:35.730759 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f3d749-a110-4129-98b9-2fb15202f562 admin admin] PATH_INFO: `/v3/users/0153773c94dd40c49dd740f27a821dfe` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:35.732804 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f3d749-a110-4129-98b9-2fb15202f562 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0153773c94dd40c49dd740f27a821dfe)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:35.735408 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f3d749-a110-4129-98b9-2fb15202f562 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:35.778866 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 817/1648] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:40:35 2022] POST /identity/v3/projects => generated 403 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:40:35.778866 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c00c3-ff7a-4604-be60-aba78f6f33df None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:35.778866 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c00c3-ff7a-4604-be60-aba78f6f33df None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:35.778866 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c00c3-ff7a-4604-be60-aba78f6f33df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:35.892589 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d5f3d749-a110-4129-98b9-2fb15202f562 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0153773c94dd40c49dd740f27a821dfe'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:35.928713 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d5f3d749-a110-4129-98b9-2fb15202f562 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0153773c94dd40c49dd740f27a821dfe'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:35.964873 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d5f3d749-a110-4129-98b9-2fb15202f562 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0153773c94dd40c49dd740f27a821dfe'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:35.967011 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d5f3d749-a110-4129-98b9-2fb15202f562 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0153773c94dd40c49dd740f27a821dfe'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:35.968907 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d5f3d749-a110-4129-98b9-2fb15202f562 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0153773c94dd40c49dd740f27a821dfe'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:36.006924 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 832/1649] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:40:35 2022] DELETE /identity/v3/users/0153773c94dd40c49dd740f27a821dfe => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:40:36.007946 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f3d749-a110-4129-98b9-2fb15202f562 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:36.013585 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-d5f3d749-a110-4129-98b9-2fb15202f562 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:36.116852 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:36.227119 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-660c00c3-ff7a-4604-be60-aba78f6f33df None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:36.227802 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f3d749-a110-4129-98b9-2fb15202f562 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:36.234631 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-660c00c3-ff7a-4604-be60-aba78f6f33df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:36.238421 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce25e20-6c64-4d36-a4c9-7e3c60a5aa06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:36.239637 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce25e20-6c64-4d36-a4c9-7e3c60a5aa06 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:36.240070 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce25e20-6c64-4d36-a4c9-7e3c60a5aa06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:36.241238 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce25e20-6c64-4d36-a4c9-7e3c60a5aa06 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:36.242235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce25e20-6c64-4d36-a4c9-7e3c60a5aa06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:36.244376 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce25e20-6c64-4d36-a4c9-7e3c60a5aa06 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:36.266312 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 818/1650] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 486 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:36.266312 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb16f1cf-aadd-44b6-8d39-d93020262b77 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:36.266312 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb16f1cf-aadd-44b6-8d39-d93020262b77 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:36.266312 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb16f1cf-aadd-44b6-8d39-d93020262b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:36.281838 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 833/1651] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:40:36 2022] POST /identity/v3/users => generated 308 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:40:36.286644 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce25e20-6c64-4d36-a4c9-7e3c60a5aa06 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:36.300611 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-bb16f1cf-aadd-44b6-8d39-d93020262b77 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:36.302552 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce25e20-6c64-4d36-a4c9-7e3c60a5aa06 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:36.305157 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-bb16f1cf-aadd-44b6-8d39-d93020262b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:36.322230 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 819/1652] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:36.324497 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb16f1cf-aadd-44b6-8d39-d93020262b77 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:36.326748 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c284a4a0-e0ac-4d26-b3c8-789ce7924906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:36.327562 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c284a4a0-e0ac-4d26-b3c8-789ce7924906 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:36.327838 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c284a4a0-e0ac-4d26-b3c8-789ce7924906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:36.328130 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c284a4a0-e0ac-4d26-b3c8-789ce7924906 admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:36.328855 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c284a4a0-e0ac-4d26-b3c8-789ce7924906 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:40:36.329160 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c284a4a0-e0ac-4d26-b3c8-789ce7924906 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:36.330899 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c284a4a0-e0ac-4d26-b3c8-789ce7924906 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:36.346132 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb16f1cf-aadd-44b6-8d39-d93020262b77 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:36.353150 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 834/1653] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:40:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:40:36.354258 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c284a4a0-e0ac-4d26-b3c8-789ce7924906 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:36.356661 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-c284a4a0-e0ac-4d26-b3c8-789ce7924906 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:36.366572 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bfdc2d-4858-4be2-b1fe-de0078666fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:36.367408 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bfdc2d-4858-4be2-b1fe-de0078666fa4 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:36.367803 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bfdc2d-4858-4be2-b1fe-de0078666fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:36.368166 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bfdc2d-4858-4be2-b1fe-de0078666fa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:36.369281 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bfdc2d-4858-4be2-b1fe-de0078666fa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:36.371140 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bfdc2d-4858-4be2-b1fe-de0078666fa4 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:36.395691 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 820/1654] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:40:36 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:40:36.396951 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bfdc2d-4858-4be2-b1fe-de0078666fa4 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:36.416104 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bfdc2d-4858-4be2-b1fe-de0078666fa4 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:36.423332 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a025fbc5-53db-40c4-8263-23efdb8d27fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:36.424004 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025fbc5-53db-40c4-8263-23efdb8d27fd admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:36.424291 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025fbc5-53db-40c4-8263-23efdb8d27fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:36.424582 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025fbc5-53db-40c4-8263-23efdb8d27fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:36.425264 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a025fbc5-53db-40c4-8263-23efdb8d27fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:36.428957 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a025fbc5-53db-40c4-8263-23efdb8d27fd admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:36.480845 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 821/1655] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:40:36 2022] POST /identity/v3/projects => generated 379 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:40:36.482240 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a025fbc5-53db-40c4-8263-23efdb8d27fd admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:36.489556 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c284a4a0-e0ac-4d26-b3c8-789ce7924906 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:36.594647 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:36.597041 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa64c06-5740-4570-aceb-39ab1338a941 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:36.597823 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa64c06-5740-4570-aceb-39ab1338a941 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:36.598111 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa64c06-5740-4570-aceb-39ab1338a941 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:36.598391 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa64c06-5740-4570-aceb-39ab1338a941 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:36.614397 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a025fbc5-53db-40c4-8263-23efdb8d27fd admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:36.624359 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d7b70e-bfe1-42f6-8f64-fb6711412e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:36.625745 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d7b70e-bfe1-42f6-8f64-fb6711412e8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:36.626253 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d7b70e-bfe1-42f6-8f64-fb6711412e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:36.626837 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d7b70e-bfe1-42f6-8f64-fb6711412e8e admin admin] PATH_INFO: `/v3/projects/c3c13f69c6294c4eab7412b91fc5fd74/users/2f84b40e57bb4ef98ac50f4f19dca379/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:36.627302 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa64c06-5740-4570-aceb-39ab1338a941 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:36.633644 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa64c06-5740-4570-aceb-39ab1338a941 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:36.650913 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d7b70e-bfe1-42f6-8f64-fb6711412e8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3c13f69c6294c4eab7412b91fc5fd74, user_id=2f84b40e57bb4ef98ac50f4f19dca379, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:36.656772 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d7b70e-bfe1-42f6-8f64-fb6711412e8e admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:36.662217 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 835/1656] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:40:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:40:36.664551 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa64c06-5740-4570-aceb-39ab1338a941 service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:36.688801 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 822/1657] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:40:36 2022] PUT /identity/v3/projects/c3c13f69c6294c4eab7412b91fc5fd74/users/2f84b40e57bb4ef98ac50f4f19dca379/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:40:36.689833 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa64c06-5740-4570-aceb-39ab1338a941 service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:36.692764 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d7b70e-bfe1-42f6-8f64-fb6711412e8e admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:36.886483 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d83084-33c0-431a-9518-f9d45ac84870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:36.887346 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d7b70e-bfe1-42f6-8f64-fb6711412e8e admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:36.889371 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d83084-33c0-431a-9518-f9d45ac84870 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:36.889371 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d83084-33c0-431a-9518-f9d45ac84870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:36.891065 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d83084-33c0-431a-9518-f9d45ac84870 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:36.892924 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d83084-33c0-431a-9518-f9d45ac84870 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:36.903707 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5aa6a6-44fd-4573-aa9c-59555da642bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:36.905548 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d83084-33c0-431a-9518-f9d45ac84870 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:36.906444 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5aa6a6-44fd-4573-aa9c-59555da642bb admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:36.907363 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5aa6a6-44fd-4573-aa9c-59555da642bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:36.908536 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5aa6a6-44fd-4573-aa9c-59555da642bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:36.909941 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5aa6a6-44fd-4573-aa9c-59555da642bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:36.915545 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5aa6a6-44fd-4573-aa9c-59555da642bb admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:36.939284 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:36.950032 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 823/1658] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:40:36 2022] GET /identity/v3/roles => generated 1420 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:40:36.954723 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5aa6a6-44fd-4573-aa9c-59555da642bb admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:36.971450 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 836/1659] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:40:36 2022] POST /identity/v3/users => generated 290 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:40:36.976388 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d83084-33c0-431a-9518-f9d45ac84870 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:36.991761 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5aa6a6-44fd-4573-aa9c-59555da642bb admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:37.009407 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7aed8b-39e2-459a-9646-c205d1416a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:37.010218 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d83084-33c0-431a-9518-f9d45ac84870 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:37.011343 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7aed8b-39e2-459a-9646-c205d1416a7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:37.012508 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7aed8b-39e2-459a-9646-c205d1416a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:37.013848 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7aed8b-39e2-459a-9646-c205d1416a7b admin admin] PATH_INFO: `/v3/projects/c3c13f69c6294c4eab7412b91fc5fd74/users/2f84b40e57bb4ef98ac50f4f19dca379/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:37.019513 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7aed8b-39e2-459a-9646-c205d1416a7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3c13f69c6294c4eab7412b91fc5fd74, user_id=2f84b40e57bb4ef98ac50f4f19dca379, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:37.034418 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24dd03e-91ed-40e1-86cb-e492bb65aaef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:37.036920 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7aed8b-39e2-459a-9646-c205d1416a7b admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:37.038886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24dd03e-91ed-40e1-86cb-e492bb65aaef admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:37.039826 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24dd03e-91ed-40e1-86cb-e492bb65aaef admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:37.040219 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24dd03e-91ed-40e1-86cb-e492bb65aaef admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:37.042723 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24dd03e-91ed-40e1-86cb-e492bb65aaef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:37.047365 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24dd03e-91ed-40e1-86cb-e492bb65aaef admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:37.070971 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 824/1660] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:40:36 2022] PUT /identity/v3/projects/c3c13f69c6294c4eab7412b91fc5fd74/users/2f84b40e57bb4ef98ac50f4f19dca379/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:40:37.111648 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 837/1661] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:40:36 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:40:37.122018 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7aed8b-39e2-459a-9646-c205d1416a7b admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:37.333489 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7aed8b-39e2-459a-9646-c205d1416a7b admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:37.340217 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00889695-85c8-4bae-a3ab-0a7172c3b675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:37.340924 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00889695-85c8-4bae-a3ab-0a7172c3b675 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:37.341212 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00889695-85c8-4bae-a3ab-0a7172c3b675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:37.341548 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00889695-85c8-4bae-a3ab-0a7172c3b675 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:37.342470 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00889695-85c8-4bae-a3ab-0a7172c3b675 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:37.344313 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00889695-85c8-4bae-a3ab-0a7172c3b675 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:37.358120 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 825/1662] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:40:37 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:40:37.359668 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24dd03e-91ed-40e1-86cb-e492bb65aaef admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:37.361786 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b24dd03e-91ed-40e1-86cb-e492bb65aaef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:37.370634 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00889695-85c8-4bae-a3ab-0a7172c3b675 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:37.386539 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24dd03e-91ed-40e1-86cb-e492bb65aaef admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:37.393635 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00889695-85c8-4bae-a3ab-0a7172c3b675 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:37.398698 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb57257-ddcd-4185-91e4-6893cba661f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:37.400481 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb57257-ddcd-4185-91e4-6893cba661f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:37.401741 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb57257-ddcd-4185-91e4-6893cba661f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:37.402022 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb57257-ddcd-4185-91e4-6893cba661f3 admin admin] PATH_INFO: `/v3/projects/d1a4e12b8f724e27ad10bf67d10bbb1e` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:37.403793 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb57257-ddcd-4185-91e4-6893cba661f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1a4e12b8f724e27ad10bf67d10bbb1e)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:37.407420 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb57257-ddcd-4185-91e4-6893cba661f3 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:37.408957 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c16cc91-da27-4813-8006-8363c62baf62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:37.410274 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16cc91-da27-4813-8006-8363c62baf62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:37.410587 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16cc91-da27-4813-8006-8363c62baf62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:37.410939 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16cc91-da27-4813-8006-8363c62baf62 admin admin] PATH_INFO: `/v3/projects/2ab3fbea9d924850b2d282c31c874aae/users/53a314460e6b4a1b9b9a78059e8bcd9e/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:37.447702 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c16cc91-da27-4813-8006-8363c62baf62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ab3fbea9d924850b2d282c31c874aae, user_id=53a314460e6b4a1b9b9a78059e8bcd9e, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:37.453900 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c16cc91-da27-4813-8006-8363c62baf62 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:37.500509 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 826/1663] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:40:37 2022] PUT /identity/v3/projects/2ab3fbea9d924850b2d282c31c874aae/users/53a314460e6b4a1b9b9a78059e8bcd9e/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:40:37.513078 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3effd00e-3f23-4225-9a2f-22edb89d70bd None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:37.513397 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3effd00e-3f23-4225-9a2f-22edb89d70bd None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:37.513630 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3effd00e-3f23-4225-9a2f-22edb89d70bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:37.549039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-bbb57257-ddcd-4185-91e4-6893cba661f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1a4e12b8f724e27ad10bf67d10bbb1e', 'request_id': 'req-bbb57257-ddcd-4185-91e4-6893cba661f3'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:37.571466 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-bbb57257-ddcd-4185-91e4-6893cba661f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1a4e12b8f724e27ad10bf67d10bbb1e', 'request_id': 'req-bbb57257-ddcd-4185-91e4-6893cba661f3'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:37.586127 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-3effd00e-3f23-4225-9a2f-22edb89d70bd None None] MFA Rules not processed for user `53a314460e6b4a1b9b9a78059e8bcd9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:37.587538 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-3effd00e-3f23-4225-9a2f-22edb89d70bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:37.595501 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-bbb57257-ddcd-4185-91e4-6893cba661f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1a4e12b8f724e27ad10bf67d10bbb1e', 'request_id': 'req-bbb57257-ddcd-4185-91e4-6893cba661f3'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:37.638044 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-bbb57257-ddcd-4185-91e4-6893cba661f3 admin admin] The token cache is being invalidate because project d1a4e12b8f724e27ad10bf67d10bbb1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:40:37.638623 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-bbb57257-ddcd-4185-91e4-6893cba661f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:37.639877 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-bbb57257-ddcd-4185-91e4-6893cba661f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:37.644689 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 838/1664] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:40:37 2022] DELETE /identity/v3/projects/d1a4e12b8f724e27ad10bf67d10bbb1e => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:40:37.651125 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e109cf-6c5f-41cf-9bb0-297e5ea67d5b None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:37.651420 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e109cf-6c5f-41cf-9bb0-297e5ea67d5b None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:37.653124 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e109cf-6c5f-41cf-9bb0-297e5ea67d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:37.660014 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:37.714545 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-b5e109cf-6c5f-41cf-9bb0-297e5ea67d5b None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:37.716796 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b5e109cf-6c5f-41cf-9bb0-297e5ea67d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:37.849349 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 827/1665] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:37 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:37.870133 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3effd00e-3f23-4225-9a2f-22edb89d70bd None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:37.875624 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-3effd00e-3f23-4225-9a2f-22edb89d70bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:37.913836 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 839/1666] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:37.920203 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50847a63-bfb9-48e8-a37c-60086213da71 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:37.920502 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50847a63-bfb9-48e8-a37c-60086213da71 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:37.920929 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50847a63-bfb9-48e8-a37c-60086213da71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:37.972340 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-50847a63-bfb9-48e8-a37c-60086213da71 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:37.973601 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-50847a63-bfb9-48e8-a37c-60086213da71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:38.001869 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 840/1667] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:38.003651 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50847a63-bfb9-48e8-a37c-60086213da71 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:38.024526 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50847a63-bfb9-48e8-a37c-60086213da71 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:38.031935 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3effd00e-3f23-4225-9a2f-22edb89d70bd None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:38.044363 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02504278-9ceb-4c5a-9648-645539d4fd4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:38.045083 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02504278-9ceb-4c5a-9648-645539d4fd4a admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:38.045351 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02504278-9ceb-4c5a-9648-645539d4fd4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:38.045606 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02504278-9ceb-4c5a-9648-645539d4fd4a admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:38.046677 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2435e03-ddb1-4e22-b528-506bc0d68343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:38.047359 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2435e03-ddb1-4e22-b528-506bc0d68343 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:38.047636 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2435e03-ddb1-4e22-b528-506bc0d68343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:38.047904 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2435e03-ddb1-4e22-b528-506bc0d68343 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:38.048606 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2435e03-ddb1-4e22-b528-506bc0d68343 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:38.050719 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2435e03-ddb1-4e22-b528-506bc0d68343 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:38.054470 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02504278-9ceb-4c5a-9648-645539d4fd4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:40:38.054799 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02504278-9ceb-4c5a-9648-645539d4fd4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:38.056893 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02504278-9ceb-4c5a-9648-645539d4fd4a admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:38.091163 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 841/1668] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:40:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:40:38.093347 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02504278-9ceb-4c5a-9648-645539d4fd4a admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:38.100988 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 828/1669] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:40:37 2022] POST /identity/v3/projects => generated 379 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:40:38.101737 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2435e03-ddb1-4e22-b528-506bc0d68343 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:38.141377 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02504278-9ceb-4c5a-9648-645539d4fd4a admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:38.297790 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2435e03-ddb1-4e22-b528-506bc0d68343 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:38.312106 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aa62b8-4bdb-4713-aa23-b1770f171299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:38.312806 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa62b8-4bdb-4713-aa23-b1770f171299 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:38.313092 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa62b8-4bdb-4713-aa23-b1770f171299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:38.313369 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa62b8-4bdb-4713-aa23-b1770f171299 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:38.314107 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aa62b8-4bdb-4713-aa23-b1770f171299 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:38.316131 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aa62b8-4bdb-4713-aa23-b1770f171299 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:38.379287 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 829/1670] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:40:38 2022] POST /identity/v3/users => generated 296 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:40:38.380313 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aa62b8-4bdb-4713-aa23-b1770f171299 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:38.387833 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-83aa62b8-4bdb-4713-aa23-b1770f171299 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:38.415164 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aa62b8-4bdb-4713-aa23-b1770f171299 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:38.429165 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a35d46a-85fd-4d8b-a575-801f5ca11c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:38.430076 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a35d46a-85fd-4d8b-a575-801f5ca11c26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:38.430361 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a35d46a-85fd-4d8b-a575-801f5ca11c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:38.430640 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a35d46a-85fd-4d8b-a575-801f5ca11c26 admin admin] PATH_INFO: `/v3/projects/96a995413a5146d88dd2d817961a9310` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:38.437830 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a35d46a-85fd-4d8b-a575-801f5ca11c26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96a995413a5146d88dd2d817961a9310)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:38.440715 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a35d46a-85fd-4d8b-a575-801f5ca11c26 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:38.529460 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:38.586350 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7a35d46a-85fd-4d8b-a575-801f5ca11c26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96a995413a5146d88dd2d817961a9310', 'request_id': 'req-7a35d46a-85fd-4d8b-a575-801f5ca11c26'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:38.607602 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:38.608117 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7a35d46a-85fd-4d8b-a575-801f5ca11c26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96a995413a5146d88dd2d817961a9310', 'request_id': 'req-7a35d46a-85fd-4d8b-a575-801f5ca11c26'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:38.631735 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7a35d46a-85fd-4d8b-a575-801f5ca11c26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96a995413a5146d88dd2d817961a9310', 'request_id': 'req-7a35d46a-85fd-4d8b-a575-801f5ca11c26'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:38.645983 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86be14-a0d5-4ff0-b0b1-89836838d76a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:38.654294 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86be14-a0d5-4ff0-b0b1-89836838d76a admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:38.656410 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86be14-a0d5-4ff0-b0b1-89836838d76a admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:38.657683 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86be14-a0d5-4ff0-b0b1-89836838d76a admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:38.660074 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c86be14-a0d5-4ff0-b0b1-89836838d76a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:38.664241 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c86be14-a0d5-4ff0-b0b1-89836838d76a admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:38.666232 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7a35d46a-85fd-4d8b-a575-801f5ca11c26 admin admin] The token cache is being invalidate because project 96a995413a5146d88dd2d817961a9310 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:40:38.667089 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7a35d46a-85fd-4d8b-a575-801f5ca11c26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:38.668561 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7a35d46a-85fd-4d8b-a575-801f5ca11c26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:38.673636 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 830/1671] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:40:38 2022] DELETE /identity/v3/projects/96a995413a5146d88dd2d817961a9310 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:40:38.674857 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a35d46a-85fd-4d8b-a575-801f5ca11c26 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:38.678078 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-7a35d46a-85fd-4d8b-a575-801f5ca11c26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:38.725594 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 842/1672] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:40:38 2022] POST /identity/v3/projects => generated 363 bytes in 630 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:40:38.725594 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73da00f-a676-40d1-a97e-17519be87119 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:38.725594 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73da00f-a676-40d1-a97e-17519be87119 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:38.725594 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73da00f-a676-40d1-a97e-17519be87119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:38.755736 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-a73da00f-a676-40d1-a97e-17519be87119 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:38.757596 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-a73da00f-a676-40d1-a97e-17519be87119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:38.869906 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a35d46a-85fd-4d8b-a575-801f5ca11c26 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:38.926832 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 843/1673] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:38.927886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73da00f-a676-40d1-a97e-17519be87119 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:38.933406 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-a73da00f-a676-40d1-a97e-17519be87119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:38.957388 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73da00f-a676-40d1-a97e-17519be87119 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:38.973017 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2def549-626e-4b6d-a6c9-f38353e914b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:38.974076 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2def549-626e-4b6d-a6c9-f38353e914b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:38.974076 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2def549-626e-4b6d-a6c9-f38353e914b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:38.975291 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2def549-626e-4b6d-a6c9-f38353e914b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:38.975691 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2def549-626e-4b6d-a6c9-f38353e914b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:38.977715 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2def549-626e-4b6d-a6c9-f38353e914b3 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:39.026175 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 844/1674] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:40:38 2022] POST /identity/v3/users => generated 282 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:40:39.030364 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1139b45-81f7-403c-a6c5-f08c54c5a132 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:39.030364 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1139b45-81f7-403c-a6c5-f08c54c5a132 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:39.030364 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1139b45-81f7-403c-a6c5-f08c54c5a132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:39.047014 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94662eb-4661-493b-8623-dcb5a1dca709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:39.049913 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94662eb-4661-493b-8623-dcb5a1dca709 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:39.050471 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94662eb-4661-493b-8623-dcb5a1dca709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:39.055247 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94662eb-4661-493b-8623-dcb5a1dca709 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:39.056905 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94662eb-4661-493b-8623-dcb5a1dca709 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:39.058965 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94662eb-4661-493b-8623-dcb5a1dca709 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:39.071601 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-d1139b45-81f7-403c-a6c5-f08c54c5a132 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:39.074952 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-d1139b45-81f7-403c-a6c5-f08c54c5a132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:39.077051 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 831/1675] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 403 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:40:39.086400 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94662eb-4661-493b-8623-dcb5a1dca709 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:39.102136 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 845/1676] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:39.105141 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1139b45-81f7-403c-a6c5-f08c54c5a132 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:39.120286 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94662eb-4661-493b-8623-dcb5a1dca709 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:39.123076 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1139b45-81f7-403c-a6c5-f08c54c5a132 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:39.140672 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b74536a-346e-4d7e-9af0-dc745022f760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:39.141441 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b196698a-f9c2-42ef-ad7a-4b4145141334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:39.141749 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74536a-346e-4d7e-9af0-dc745022f760 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:39.142034 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74536a-346e-4d7e-9af0-dc745022f760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:39.142312 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74536a-346e-4d7e-9af0-dc745022f760 admin admin] PATH_INFO: `/v3/projects/3280bb930e9e403894457959c81e8ba5/users/b5db2014719146c5a014d97a334c78b6/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:39.143103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b196698a-f9c2-42ef-ad7a-4b4145141334 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:39.144777 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b196698a-f9c2-42ef-ad7a-4b4145141334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:39.145301 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b196698a-f9c2-42ef-ad7a-4b4145141334 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:39.149901 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b196698a-f9c2-42ef-ad7a-4b4145141334 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:39.152246 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b196698a-f9c2-42ef-ad7a-4b4145141334 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:39.173896 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b74536a-346e-4d7e-9af0-dc745022f760 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3280bb930e9e403894457959c81e8ba5, user_id=b5db2014719146c5a014d97a334c78b6, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:39.176507 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b74536a-346e-4d7e-9af0-dc745022f760 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:39.192344 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 832/1677] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:40:39 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:40:39.202763 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b196698a-f9c2-42ef-ad7a-4b4145141334 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:39.222473 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 846/1678] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:40:39 2022] PUT /identity/v3/projects/3280bb930e9e403894457959c81e8ba5/users/b5db2014719146c5a014d97a334c78b6/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:40:39.222473 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b74536a-346e-4d7e-9af0-dc745022f760 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:39.236050 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b196698a-f9c2-42ef-ad7a-4b4145141334 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:39.313735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b74536a-346e-4d7e-9af0-dc745022f760 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:39.319262 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea41675f-a6e1-44de-b279-d59f67c887dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:39.320243 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea41675f-a6e1-44de-b279-d59f67c887dd admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:39.320803 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea41675f-a6e1-44de-b279-d59f67c887dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:39.321179 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea41675f-a6e1-44de-b279-d59f67c887dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:39.322080 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea41675f-a6e1-44de-b279-d59f67c887dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:39.324868 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea41675f-a6e1-44de-b279-d59f67c887dd admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:39.343527 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 847/1679] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:40:39 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:40:39.344622 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea41675f-a6e1-44de-b279-d59f67c887dd admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:39.364574 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea41675f-a6e1-44de-b279-d59f67c887dd admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:39.370550 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:39.382920 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cce14c1-8cea-41d2-a84c-d404b24c85b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:39.383704 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cce14c1-8cea-41d2-a84c-d404b24c85b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:39.385267 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cce14c1-8cea-41d2-a84c-d404b24c85b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:39.385624 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cce14c1-8cea-41d2-a84c-d404b24c85b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:39.386782 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cce14c1-8cea-41d2-a84c-d404b24c85b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:39.388720 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cce14c1-8cea-41d2-a84c-d404b24c85b5 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:39.401491 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 848/1680] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:40:39 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:40:39.402749 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cce14c1-8cea-41d2-a84c-d404b24c85b5 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:39.409106 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50207e09-bff1-4605-ba27-eb463db8a34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:39.409842 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50207e09-bff1-4605-ba27-eb463db8a34b admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:39.410132 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50207e09-bff1-4605-ba27-eb463db8a34b admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:39.410432 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50207e09-bff1-4605-ba27-eb463db8a34b admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:39.415470 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50207e09-bff1-4605-ba27-eb463db8a34b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:40:39.415832 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50207e09-bff1-4605-ba27-eb463db8a34b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:39.417375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cce14c1-8cea-41d2-a84c-d404b24c85b5 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:39.423327 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d58474-2665-4cb7-b744-49f16e7af8c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:39.423496 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50207e09-bff1-4605-ba27-eb463db8a34b admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:39.424792 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d58474-2665-4cb7-b744-49f16e7af8c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:39.425217 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d58474-2665-4cb7-b744-49f16e7af8c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:39.425811 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d58474-2665-4cb7-b744-49f16e7af8c5 admin admin] PATH_INFO: `/v3/projects/aabb589c9bdb4a2795ab8a13b07c89c2/users/6036b62db23a40ba9d7e4f6d7995eb01/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:39.431662 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:39.450141 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 833/1681] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:40:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:40:39.451112 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50207e09-bff1-4605-ba27-eb463db8a34b admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:39.453159 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d58474-2665-4cb7-b744-49f16e7af8c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aabb589c9bdb4a2795ab8a13b07c89c2, user_id=6036b62db23a40ba9d7e4f6d7995eb01, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:39.456006 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d58474-2665-4cb7-b744-49f16e7af8c5 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:39.475160 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50207e09-bff1-4605-ba27-eb463db8a34b admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:39.475978 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 849/1682] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:40:39 2022] PUT /identity/v3/projects/aabb589c9bdb4a2795ab8a13b07c89c2/users/6036b62db23a40ba9d7e4f6d7995eb01/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:40:39.477118 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d58474-2665-4cb7-b744-49f16e7af8c5 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:39.563492 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d58474-2665-4cb7-b744-49f16e7af8c5 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:39.571245 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd33561-1cb2-439f-9caf-04617532e9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:39.573083 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd33561-1cb2-439f-9caf-04617532e9be admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:39.573083 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd33561-1cb2-439f-9caf-04617532e9be admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:39.573259 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd33561-1cb2-439f-9caf-04617532e9be admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:39.574215 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd33561-1cb2-439f-9caf-04617532e9be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:39.576736 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd33561-1cb2-439f-9caf-04617532e9be admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:39.612789 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 850/1683] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:40:39 2022] POST /identity/v3/projects => generated 365 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:40:39.614594 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b971a-8490-4032-be36-a465446767bd None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:39.616980 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b971a-8490-4032-be36-a465446767bd None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:39.617444 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b971a-8490-4032-be36-a465446767bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:39.654779 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-ae9b971a-8490-4032-be36-a465446767bd None None] MFA Rules not processed for user `6036b62db23a40ba9d7e4f6d7995eb01`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:39.655070 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee941b8-320b-40f1-b1e4-dcb8c7bf8813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:39.656217 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee941b8-320b-40f1-b1e4-dcb8c7bf8813 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:39.656538 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee941b8-320b-40f1-b1e4-dcb8c7bf8813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:39.656914 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee941b8-320b-40f1-b1e4-dcb8c7bf8813 admin admin] PATH_INFO: `/v3/projects/3280bb930e9e403894457959c81e8ba5/users/b5db2014719146c5a014d97a334c78b6/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:39.657723 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-ae9b971a-8490-4032-be36-a465446767bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:39.660799 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee941b8-320b-40f1-b1e4-dcb8c7bf8813 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3280bb930e9e403894457959c81e8ba5, user_id=b5db2014719146c5a014d97a334c78b6, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:39.666356 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee941b8-320b-40f1-b1e4-dcb8c7bf8813 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:39.690493 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 834/1684] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:40:39 2022] PUT /identity/v3/projects/3280bb930e9e403894457959c81e8ba5/users/b5db2014719146c5a014d97a334c78b6/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:40:39.703771 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee941b8-320b-40f1-b1e4-dcb8c7bf8813 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:39.746537 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 851/1685] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:39 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:39.748686 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c398e9e0-202d-46e4-9b8f-1fa0ac6111ac None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:39.748979 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c398e9e0-202d-46e4-9b8f-1fa0ac6111ac None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:39.749261 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c398e9e0-202d-46e4-9b8f-1fa0ac6111ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:39.808756 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-c398e9e0-202d-46e4-9b8f-1fa0ac6111ac None None] MFA Rules not processed for user `b5db2014719146c5a014d97a334c78b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:39.810730 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-c398e9e0-202d-46e4-9b8f-1fa0ac6111ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:39.890578 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee941b8-320b-40f1-b1e4-dcb8c7bf8813 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:39.904014 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a819f5bc-8e25-457a-b0fa-ec1d6da127f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:39.904864 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819f5bc-8e25-457a-b0fa-ec1d6da127f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:39.905320 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819f5bc-8e25-457a-b0fa-ec1d6da127f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:39.905647 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819f5bc-8e25-457a-b0fa-ec1d6da127f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:39.906890 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:39.908258 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a819f5bc-8e25-457a-b0fa-ec1d6da127f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:39.913543 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a819f5bc-8e25-457a-b0fa-ec1d6da127f4 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:39.946820 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 852/1686] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:39 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:39.950472 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c398e9e0-202d-46e4-9b8f-1fa0ac6111ac None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:39.962670 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 835/1687] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:40:39 2022] POST /identity/v3/users => generated 283 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:40:39.963891 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a819f5bc-8e25-457a-b0fa-ec1d6da127f4 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:39.970660 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c398e9e0-202d-46e4-9b8f-1fa0ac6111ac None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:39.970856 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-a819f5bc-8e25-457a-b0fa-ec1d6da127f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:39.977219 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bc13f1-962c-47b4-b26a-f8df02d80e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:39.978245 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bc13f1-962c-47b4-b26a-f8df02d80e0e admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:39.978358 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bc13f1-962c-47b4-b26a-f8df02d80e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:39.978585 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bc13f1-962c-47b4-b26a-f8df02d80e0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:39.979749 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bc13f1-962c-47b4-b26a-f8df02d80e0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:40.010181 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bc13f1-962c-47b4-b26a-f8df02d80e0e admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:40.084778 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 853/1688] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:40:39 2022] POST /identity/v3/projects => generated 363 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:40:40.088105 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bc13f1-962c-47b4-b26a-f8df02d80e0e admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:40.125523 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a819f5bc-8e25-457a-b0fa-ec1d6da127f4 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:40.209928 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:40.273443 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02006c43-68ba-48e9-9956-d3e8b3bb995d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:40.275197 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02006c43-68ba-48e9-9956-d3e8b3bb995d service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:40.275316 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bc13f1-962c-47b4-b26a-f8df02d80e0e admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:40.275790 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02006c43-68ba-48e9-9956-d3e8b3bb995d service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:40.276210 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02006c43-68ba-48e9-9956-d3e8b3bb995d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:40.288841 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2feb077-ceb6-4dbf-ac49-59829d2844e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:40.289549 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2feb077-ceb6-4dbf-ac49-59829d2844e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:40.290001 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2feb077-ceb6-4dbf-ac49-59829d2844e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:40.290279 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2feb077-ceb6-4dbf-ac49-59829d2844e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:40.291032 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2feb077-ceb6-4dbf-ac49-59829d2844e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:40.296387 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2feb077-ceb6-4dbf-ac49-59829d2844e9 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:40.315117 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 854/1689] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:40:40 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:40:40.322435 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2feb077-ceb6-4dbf-ac49-59829d2844e9 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:40.355829 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2feb077-ceb6-4dbf-ac49-59829d2844e9 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:40.370974 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615b89a1-2999-4644-97d5-a4dfd61c5b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:40.371781 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615b89a1-2999-4644-97d5-a4dfd61c5b8f admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:40.371781 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615b89a1-2999-4644-97d5-a4dfd61c5b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:40.371993 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615b89a1-2999-4644-97d5-a4dfd61c5b8f admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:40.372901 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615b89a1-2999-4644-97d5-a4dfd61c5b8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:40.375022 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615b89a1-2999-4644-97d5-a4dfd61c5b8f admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:40.414523 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02006c43-68ba-48e9-9956-d3e8b3bb995d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:40.415883 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02006c43-68ba-48e9-9956-d3e8b3bb995d service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:40.417847 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 855/1690] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:40:40 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:40:40.418651 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615b89a1-2999-4644-97d5-a4dfd61c5b8f admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:40.437050 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 836/1691] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:40:39 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:40:40.438646 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02006c43-68ba-48e9-9956-d3e8b3bb995d service cinder] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:40.449418 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615b89a1-2999-4644-97d5-a4dfd61c5b8f admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:40.471657 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db5b42b-f4bf-4778-964f-ee9dc35257ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:40.472119 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db5b42b-f4bf-4778-964f-ee9dc35257ca admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:40.472410 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db5b42b-f4bf-4778-964f-ee9dc35257ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:40.473335 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db5b42b-f4bf-4778-964f-ee9dc35257ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:40.473449 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db5b42b-f4bf-4778-964f-ee9dc35257ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:40.491607 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db5b42b-f4bf-4778-964f-ee9dc35257ca admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:40.501435 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02006c43-68ba-48e9-9956-d3e8b3bb995d service cinder] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:40.537659 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08362943-c15a-49b4-b63e-e804f273e06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:40.538485 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08362943-c15a-49b4-b63e-e804f273e06f admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:40.538774 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08362943-c15a-49b4-b63e-e804f273e06f admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:40.539051 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08362943-c15a-49b4-b63e-e804f273e06f admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:40.539792 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08362943-c15a-49b4-b63e-e804f273e06f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:40.541415 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08362943-c15a-49b4-b63e-e804f273e06f admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:40.564529 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 837/1692] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:40:40 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:40:40.565855 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 856/1693] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:40:40 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:40:40.568796 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db5b42b-f4bf-4778-964f-ee9dc35257ca admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:40.570747 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08362943-c15a-49b4-b63e-e804f273e06f admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:40.601103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08362943-c15a-49b4-b63e-e804f273e06f admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:40.606826 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db5b42b-f4bf-4778-964f-ee9dc35257ca admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:40.619996 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afacc30-33a5-488b-99da-b4c52a10218a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:40.620934 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afacc30-33a5-488b-99da-b4c52a10218a admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:40.621219 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afacc30-33a5-488b-99da-b4c52a10218a admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:40.621501 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afacc30-33a5-488b-99da-b4c52a10218a admin admin] PATH_INFO: `/v3/projects/3c9ab5429d4b4a1fb334ed4658bdf701/users/313791abee5445dc91aeed15c20b0def/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:40.623599 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cac77db-bdf4-4a05-895b-bf8cd72a0e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:40.624505 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac77db-bdf4-4a05-895b-bf8cd72a0e47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:40.624785 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac77db-bdf4-4a05-895b-bf8cd72a0e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:40.625061 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac77db-bdf4-4a05-895b-bf8cd72a0e47 admin admin] PATH_INFO: `/v3/projects/546b8d65c8784cf4aa352b62557a517f/users/d333ec005de042a9b8fc8cdc78c96ee8/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:40.681637 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cac77db-bdf4-4a05-895b-bf8cd72a0e47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=546b8d65c8784cf4aa352b62557a517f, user_id=d333ec005de042a9b8fc8cdc78c96ee8, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:40.684447 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cac77db-bdf4-4a05-895b-bf8cd72a0e47 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:40.690747 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afacc30-33a5-488b-99da-b4c52a10218a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c9ab5429d4b4a1fb334ed4658bdf701, user_id=313791abee5445dc91aeed15c20b0def, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:40.701817 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afacc30-33a5-488b-99da-b4c52a10218a admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:40.736738 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 838/1694] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:40:40 2022] PUT /identity/v3/projects/546b8d65c8784cf4aa352b62557a517f/users/d333ec005de042a9b8fc8cdc78c96ee8/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:40:40.736941 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 857/1695] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:40:40 2022] PUT /identity/v3/projects/3c9ab5429d4b4a1fb334ed4658bdf701/users/313791abee5445dc91aeed15c20b0def/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:40:40.736941 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afacc30-33a5-488b-99da-b4c52a10218a admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:40.737121 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cac77db-bdf4-4a05-895b-bf8cd72a0e47 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:40.847014 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cac77db-bdf4-4a05-895b-bf8cd72a0e47 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:40.876379 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afacc30-33a5-488b-99da-b4c52a10218a admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:40.882980 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5a9fa3-6dae-41aa-9b19-738918fd77ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:40.883676 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a9fa3-6dae-41aa-9b19-738918fd77ce admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:40.883961 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a9fa3-6dae-41aa-9b19-738918fd77ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:40.884241 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a9fa3-6dae-41aa-9b19-738918fd77ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:40.884987 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5a9fa3-6dae-41aa-9b19-738918fd77ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:40.894494 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5a9fa3-6dae-41aa-9b19-738918fd77ce admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:40.919500 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 858/1696] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:40:40 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:40:40.921049 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8533f9-3205-428e-9adc-c4de332545c9 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:40.924438 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8533f9-3205-428e-9adc-c4de332545c9 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:40.925142 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8533f9-3205-428e-9adc-c4de332545c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:40.948328 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:40.965325 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7303e51-2bd3-48af-8705-3b89808e703a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:40.966647 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7303e51-2bd3-48af-8705-3b89808e703a service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:40.967247 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7303e51-2bd3-48af-8705-3b89808e703a service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:40.967670 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7303e51-2bd3-48af-8705-3b89808e703a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:40.971078 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-c7303e51-2bd3-48af-8705-3b89808e703a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:40.984318 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:41.007625 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-0b8533f9-3205-428e-9adc-c4de332545c9 None None] MFA Rules not processed for user `d333ec005de042a9b8fc8cdc78c96ee8`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:41.008929 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-0b8533f9-3205-428e-9adc-c4de332545c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:41.069510 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7303e51-2bd3-48af-8705-3b89808e703a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:41.071449 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7303e51-2bd3-48af-8705-3b89808e703a service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:41.084589 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 839/1697] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:40:40 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:40:41.085886 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7303e51-2bd3-48af-8705-3b89808e703a service cinder] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:41.101891 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7303e51-2bd3-48af-8705-3b89808e703a service cinder] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:41.109193 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747d7fce-e260-4421-98a3-bba3ad2375c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:41.110284 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747d7fce-e260-4421-98a3-bba3ad2375c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:41.110713 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747d7fce-e260-4421-98a3-bba3ad2375c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:41.111003 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747d7fce-e260-4421-98a3-bba3ad2375c5 admin admin] PATH_INFO: `/v3/projects/3c9ab5429d4b4a1fb334ed4658bdf701/users/313791abee5445dc91aeed15c20b0def/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:41.113183 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747d7fce-e260-4421-98a3-bba3ad2375c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c9ab5429d4b4a1fb334ed4658bdf701, user_id=313791abee5445dc91aeed15c20b0def, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:41.116884 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747d7fce-e260-4421-98a3-bba3ad2375c5 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:41.140571 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 840/1698] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:40:41 2022] PUT /identity/v3/projects/3c9ab5429d4b4a1fb334ed4658bdf701/users/313791abee5445dc91aeed15c20b0def/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:40:41.142001 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671fa5a2-1646-4a10-9e63-422a5e74b890 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:41.142330 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671fa5a2-1646-4a10-9e63-422a5e74b890 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:41.142618 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671fa5a2-1646-4a10-9e63-422a5e74b890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:41.171639 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 859/1699] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:40 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:41.173788 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca68a410-0de8-4d75-8b71-bd0e08146496 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:41.186216 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca68a410-0de8-4d75-8b71-bd0e08146496 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:41.186678 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca68a410-0de8-4d75-8b71-bd0e08146496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:41.194106 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-671fa5a2-1646-4a10-9e63-422a5e74b890 None None] MFA Rules not processed for user `75a612089b2b46a6b3ff5a6cd9e59579`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:41.200482 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-671fa5a2-1646-4a10-9e63-422a5e74b890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:41.276314 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-ca68a410-0de8-4d75-8b71-bd0e08146496 None None] MFA Rules not processed for user `313791abee5445dc91aeed15c20b0def`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:41.280955 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-ca68a410-0de8-4d75-8b71-bd0e08146496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:41.347147 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 841/1700] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:41.347147 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671fa5a2-1646-4a10-9e63-422a5e74b890 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:41.347147 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-671fa5a2-1646-4a10-9e63-422a5e74b890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:41.452689 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671fa5a2-1646-4a10-9e63-422a5e74b890 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:41.462466 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e522ada-e358-43de-a680-40c591991387 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:41.463024 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e522ada-e358-43de-a680-40c591991387 service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:41.464257 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e522ada-e358-43de-a680-40c591991387 service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:41.464648 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e522ada-e358-43de-a680-40c591991387 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:41.482425 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 860/1701] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:41 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:41.482425 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca68a410-0de8-4d75-8b71-bd0e08146496 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:41.482425 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-ca68a410-0de8-4d75-8b71-bd0e08146496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:41.573081 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:41.601896 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e522ada-e358-43de-a680-40c591991387 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:41.604163 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e522ada-e358-43de-a680-40c591991387 service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:41.616339 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca68a410-0de8-4d75-8b71-bd0e08146496 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:41.629332 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1eaf7-6360-4f04-a585-cc004331bb80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74bb1bc6821b4500b26f160ccae5b7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:41.630300 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1eaf7-6360-4f04-a585-cc004331bb80 service nova] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:41.631285 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1eaf7-6360-4f04-a585-cc004331bb80 service nova] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:41.631913 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1eaf7-6360-4f04-a585-cc004331bb80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:41.632021 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 842/1702] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:40:41 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:40:41.633219 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e522ada-e358-43de-a680-40c591991387 service cinder] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:41.649152 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e522ada-e358-43de-a680-40c591991387 service cinder] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:41.651267 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c1eaf7-6360-4f04-a585-cc004331bb80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:41.653285 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c1eaf7-6360-4f04-a585-cc004331bb80 service nova] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:41.659001 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ec6b9f-701f-4519-bfc5-d74d31fc6e08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:41.660838 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec6b9f-701f-4519-bfc5-d74d31fc6e08 service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:41.661077 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec6b9f-701f-4519-bfc5-d74d31fc6e08 service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:41.662254 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec6b9f-701f-4519-bfc5-d74d31fc6e08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:41.667603 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 861/1703] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:40:41 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:40:41.685025 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ec6b9f-701f-4519-bfc5-d74d31fc6e08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:41.689012 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ec6b9f-701f-4519-bfc5-d74d31fc6e08 service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:41.733970 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 843/1704] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:40:41 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:40:41.762900 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6c806-e550-46bb-8cc9-4bf79e2e6448 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:41.763391 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6c806-e550-46bb-8cc9-4bf79e2e6448 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:41.763861 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6c806-e550-46bb-8cc9-4bf79e2e6448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:41.772669 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:41.803403 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-8ed6c806-e550-46bb-8cc9-4bf79e2e6448 None None] MFA Rules not processed for user `2f84b40e57bb4ef98ac50f4f19dca379`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:41.805074 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-8ed6c806-e550-46bb-8cc9-4bf79e2e6448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:41.843714 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ec6b9f-701f-4519-bfc5-d74d31fc6e08 service cinder] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:41.855083 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ec6b9f-701f-4519-bfc5-d74d31fc6e08 service cinder] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:41.871480 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a16f2b-9d8c-4714-b0fe-7e01720dcad5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:41.873235 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a16f2b-9d8c-4714-b0fe-7e01720dcad5 service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:41.873809 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a16f2b-9d8c-4714-b0fe-7e01720dcad5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:41.874251 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a16f2b-9d8c-4714-b0fe-7e01720dcad5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:41.962957 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 862/1705] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:41 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:42.040381 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a16f2b-9d8c-4714-b0fe-7e01720dcad5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:42.045327 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a16f2b-9d8c-4714-b0fe-7e01720dcad5 service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:42.076353 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 844/1706] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:40:41 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:40:43.989228 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9b47c-038b-4192-b0a3-d7fa90b0f1cc None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:43.989796 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9b47c-038b-4192-b0a3-d7fa90b0f1cc None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:43.990939 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9b47c-038b-4192-b0a3-d7fa90b0f1cc None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:43.994067 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 863/1707] 10.222.0.13 () {54 vars in 921 bytes} [Fri Jun 24 08:40:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:40:44.038795 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf3f002-a0ff-4ecf-a758-52e326a22813 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:44.039356 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf3f002-a0ff-4ecf-a758-52e326a22813 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:44.039515 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf3f002-a0ff-4ecf-a758-52e326a22813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:44.096758 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-5cf3f002-a0ff-4ecf-a758-52e326a22813 None None] MFA Rules not processed for user `39cc268bad604dd08594e3b39c7ecf5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:44.098042 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-5cf3f002-a0ff-4ecf-a758-52e326a22813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:44.344833 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 845/1708] 10.222.0.13 () {58 vars in 1018 bytes} [Fri Jun 24 08:40:44 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:40:44.416444 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae1fa0-4e11-4799-828b-9cc1c0dc1617 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:44.416444 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae1fa0-4e11-4799-828b-9cc1c0dc1617 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:44.416444 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae1fa0-4e11-4799-828b-9cc1c0dc1617 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:44.416444 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 864/1709] 10.222.0.13 () {54 vars in 960 bytes} [Fri Jun 24 08:40:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:40:44.426092 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6c72c-2534-4532-a975-7c1dd7861f4a None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:44.426092 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6c72c-2534-4532-a975-7c1dd7861f4a None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:44.426092 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6c72c-2534-4532-a975-7c1dd7861f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:44.462061 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-91e6c72c-2534-4532-a975-7c1dd7861f4a None None] MFA Rules not processed for user `39cc268bad604dd08594e3b39c7ecf5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:44.463799 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-91e6c72c-2534-4532-a975-7c1dd7861f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:44.495378 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ae1fa0-4e11-4799-828b-9cc1c0dc1617 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:44.511459 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 846/1710] 10.222.0.13 () {58 vars in 1057 bytes} [Fri Jun 24 08:40:44 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:40:44.532231 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ae1fa0-4e11-4799-828b-9cc1c0dc1617 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:44.543510 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255a78a0-9144-48e7-85c9-8cc87b69e22e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39cc268bad604dd08594e3b39c7ecf5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:44.545626 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255a78a0-9144-48e7-85c9-8cc87b69e22e service placement] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:44.546929 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255a78a0-9144-48e7-85c9-8cc87b69e22e service placement] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:44.547576 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255a78a0-9144-48e7-85c9-8cc87b69e22e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:44.563847 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255a78a0-9144-48e7-85c9-8cc87b69e22e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:44.566667 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255a78a0-9144-48e7-85c9-8cc87b69e22e service placement] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:44.598972 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 865/1711] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:40:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:40:46.201020 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e6c72c-2534-4532-a975-7c1dd7861f4a None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:46.405652 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:46.443721 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e6c72c-2534-4532-a975-7c1dd7861f4a None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:46.452051 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5697b93-1827-4cd5-9eb6-e017bcc25a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:46.454225 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5697b93-1827-4cd5-9eb6-e017bcc25a58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:46.454552 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5697b93-1827-4cd5-9eb6-e017bcc25a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:46.455004 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5697b93-1827-4cd5-9eb6-e017bcc25a58 admin admin] PATH_INFO: `/v3/users/6036b62db23a40ba9d7e4f6d7995eb01` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:46.456860 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5697b93-1827-4cd5-9eb6-e017bcc25a58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6036b62db23a40ba9d7e4f6d7995eb01)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:46.463062 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5697b93-1827-4cd5-9eb6-e017bcc25a58 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:46.579457 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f5697b93-1827-4cd5-9eb6-e017bcc25a58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6036b62db23a40ba9d7e4f6d7995eb01'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:46.634582 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f5697b93-1827-4cd5-9eb6-e017bcc25a58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6036b62db23a40ba9d7e4f6d7995eb01'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:46.637486 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f5697b93-1827-4cd5-9eb6-e017bcc25a58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6036b62db23a40ba9d7e4f6d7995eb01'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:46.659341 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f5697b93-1827-4cd5-9eb6-e017bcc25a58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6036b62db23a40ba9d7e4f6d7995eb01'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:46.670256 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f5697b93-1827-4cd5-9eb6-e017bcc25a58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6036b62db23a40ba9d7e4f6d7995eb01'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:46.696838 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 847/1712] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:40:46 2022] DELETE /identity/v3/users/6036b62db23a40ba9d7e4f6d7995eb01 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:40:46.702517 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e350b-e1b1-43fe-918e-0bb62886de0e None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:46.703108 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e350b-e1b1-43fe-918e-0bb62886de0e None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:46.703108 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e350b-e1b1-43fe-918e-0bb62886de0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:46.754743 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-107e350b-e1b1-43fe-918e-0bb62886de0e None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:46.756187 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-107e350b-e1b1-43fe-918e-0bb62886de0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:46.963573 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 866/1713] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:46.967941 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5697b93-1827-4cd5-9eb6-e017bcc25a58 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:46.969771 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-f5697b93-1827-4cd5-9eb6-e017bcc25a58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:47.121563 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5697b93-1827-4cd5-9eb6-e017bcc25a58 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:47.126071 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca0f452-d1d2-49ef-841a-d172c009cb71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:47.127040 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0f452-d1d2-49ef-841a-d172c009cb71 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:47.127416 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0f452-d1d2-49ef-841a-d172c009cb71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:47.128228 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0f452-d1d2-49ef-841a-d172c009cb71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:47.145984 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca0f452-d1d2-49ef-841a-d172c009cb71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:47.147887 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca0f452-d1d2-49ef-841a-d172c009cb71 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:47.186954 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 848/1714] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:40:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:40:47.716785 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107e350b-e1b1-43fe-918e-0bb62886de0e None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:47.719586 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-107e350b-e1b1-43fe-918e-0bb62886de0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:47.741944 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107e350b-e1b1-43fe-918e-0bb62886de0e None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:47.748325 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5b0f39-5424-4cce-a028-9b41aecf3643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:47.749670 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5b0f39-5424-4cce-a028-9b41aecf3643 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:47.750314 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5b0f39-5424-4cce-a028-9b41aecf3643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:47.751461 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5b0f39-5424-4cce-a028-9b41aecf3643 admin admin] PATH_INFO: `/v3/projects/aabb589c9bdb4a2795ab8a13b07c89c2` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:47.755517 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5b0f39-5424-4cce-a028-9b41aecf3643 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aabb589c9bdb4a2795ab8a13b07c89c2)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:47.758955 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5b0f39-5424-4cce-a028-9b41aecf3643 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:47.876426 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0f5b0f39-5424-4cce-a028-9b41aecf3643 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aabb589c9bdb4a2795ab8a13b07c89c2', 'request_id': 'req-0f5b0f39-5424-4cce-a028-9b41aecf3643'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:47.892964 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0f5b0f39-5424-4cce-a028-9b41aecf3643 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aabb589c9bdb4a2795ab8a13b07c89c2', 'request_id': 'req-0f5b0f39-5424-4cce-a028-9b41aecf3643'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:47.905239 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0f5b0f39-5424-4cce-a028-9b41aecf3643 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aabb589c9bdb4a2795ab8a13b07c89c2', 'request_id': 'req-0f5b0f39-5424-4cce-a028-9b41aecf3643'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:47.947982 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:47.951630 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0f5b0f39-5424-4cce-a028-9b41aecf3643 admin admin] The token cache is being invalidate because project aabb589c9bdb4a2795ab8a13b07c89c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:40:47.952023 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0f5b0f39-5424-4cce-a028-9b41aecf3643 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:47.954383 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0f5b0f39-5424-4cce-a028-9b41aecf3643 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:47.959585 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 867/1715] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:40:47 2022] DELETE /identity/v3/projects/aabb589c9bdb4a2795ab8a13b07c89c2 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:40:47.963330 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca0f452-d1d2-49ef-841a-d172c009cb71 service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:47.982088 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca0f452-d1d2-49ef-841a-d172c009cb71 service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:48.125175 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f307fda2-6a95-4413-9a71-ae027dbfe4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:48.126406 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f307fda2-6a95-4413-9a71-ae027dbfe4a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:48.128471 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f307fda2-6a95-4413-9a71-ae027dbfe4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:48.129143 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f307fda2-6a95-4413-9a71-ae027dbfe4a9 admin admin] PATH_INFO: `/v3/users/313791abee5445dc91aeed15c20b0def` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:48.135940 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f307fda2-6a95-4413-9a71-ae027dbfe4a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=313791abee5445dc91aeed15c20b0def)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:48.139142 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f307fda2-6a95-4413-9a71-ae027dbfe4a9 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:48.160207 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:48.183882 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f49f0-0cf8-4226-976d-5968dcb400f8 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:48.184735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f49f0-0cf8-4226-976d-5968dcb400f8 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:48.185163 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f49f0-0cf8-4226-976d-5968dcb400f8 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:48.187495 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 868/1716] 10.222.0.13 () {54 vars in 921 bytes} [Fri Jun 24 08:40:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:40:48.218895 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c60027-79ce-4bab-b432-e002ba3d1c3a None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:48.219521 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c60027-79ce-4bab-b432-e002ba3d1c3a None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:48.219959 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c60027-79ce-4bab-b432-e002ba3d1c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:48.226028 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f307fda2-6a95-4413-9a71-ae027dbfe4a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '313791abee5445dc91aeed15c20b0def'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:48.253595 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f307fda2-6a95-4413-9a71-ae027dbfe4a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '313791abee5445dc91aeed15c20b0def'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:48.254201 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f307fda2-6a95-4413-9a71-ae027dbfe4a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '313791abee5445dc91aeed15c20b0def'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:48.265957 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f307fda2-6a95-4413-9a71-ae027dbfe4a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '313791abee5445dc91aeed15c20b0def'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:48.267600 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f307fda2-6a95-4413-9a71-ae027dbfe4a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '313791abee5445dc91aeed15c20b0def'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:48.273097 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-15c60027-79ce-4bab-b432-e002ba3d1c3a None None] MFA Rules not processed for user `39cc268bad604dd08594e3b39c7ecf5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:48.274538 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-15c60027-79ce-4bab-b432-e002ba3d1c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:48.289066 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 849/1717] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:40:47 2022] DELETE /identity/v3/users/313791abee5445dc91aeed15c20b0def => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:40:48.455606 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 869/1718] 10.222.0.13 () {58 vars in 1018 bytes} [Fri Jun 24 08:40:48 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:40:48.462542 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f307fda2-6a95-4413-9a71-ae027dbfe4a9 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:48.465434 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-f307fda2-6a95-4413-9a71-ae027dbfe4a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:48.484005 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c60027-79ce-4bab-b432-e002ba3d1c3a None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:48.485480 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-15c60027-79ce-4bab-b432-e002ba3d1c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:48.521954 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c60027-79ce-4bab-b432-e002ba3d1c3a None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:48.643495 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f307fda2-6a95-4413-9a71-ae027dbfe4a9 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:48.665874 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb085473-3ce2-49e0-9589-a0090509b352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:48.667963 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb085473-3ce2-49e0-9589-a0090509b352 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:48.676661 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb085473-3ce2-49e0-9589-a0090509b352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:48.677611 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb085473-3ce2-49e0-9589-a0090509b352 admin admin] PATH_INFO: `/v3/users/b8001fbcde0244089c3e6f999b339464` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:48.687728 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb085473-3ce2-49e0-9589-a0090509b352 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8001fbcde0244089c3e6f999b339464)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:48.690460 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb085473-3ce2-49e0-9589-a0090509b352 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:48.707629 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9359574d-8b45-4f37-858f-de9e5a0747f0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39cc268bad604dd08594e3b39c7ecf5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:48.708318 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9359574d-8b45-4f37-858f-de9e5a0747f0 service placement] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:48.709311 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9359574d-8b45-4f37-858f-de9e5a0747f0 service placement] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:48.709742 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9359574d-8b45-4f37-858f-de9e5a0747f0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:48.732837 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9359574d-8b45-4f37-858f-de9e5a0747f0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:48.735544 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9359574d-8b45-4f37-858f-de9e5a0747f0 service placement] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:48.766982 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 870/1719] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:40:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:40:48.788825 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-fb085473-3ce2-49e0-9589-a0090509b352 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8001fbcde0244089c3e6f999b339464'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:48.819434 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-fb085473-3ce2-49e0-9589-a0090509b352 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8001fbcde0244089c3e6f999b339464'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:48.820533 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-fb085473-3ce2-49e0-9589-a0090509b352 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8001fbcde0244089c3e6f999b339464'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:48.837485 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-fb085473-3ce2-49e0-9589-a0090509b352 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8001fbcde0244089c3e6f999b339464'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:48.839037 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-fb085473-3ce2-49e0-9589-a0090509b352 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8001fbcde0244089c3e6f999b339464'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:48.887772 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 850/1720] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:40:48 2022] DELETE /identity/v3/users/b8001fbcde0244089c3e6f999b339464 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:40:49.037530 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f52ba0-106b-4ada-b371-2cc3f74a4408 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:49.038195 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f52ba0-106b-4ada-b371-2cc3f74a4408 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:49.038772 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f52ba0-106b-4ada-b371-2cc3f74a4408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:49.085352 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-60f52ba0-106b-4ada-b371-2cc3f74a4408 None None] MFA Rules not processed for user `71cbad05d1a54b4b9959e2ebbcb720fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:49.087863 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-60f52ba0-106b-4ada-b371-2cc3f74a4408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:49.213473 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 871/1721] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:49.229182 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb085473-3ce2-49e0-9589-a0090509b352 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:49.230941 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-fb085473-3ce2-49e0-9589-a0090509b352 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:49.274699 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f52ba0-106b-4ada-b371-2cc3f74a4408 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:49.282033 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:49.287444 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-60f52ba0-106b-4ada-b371-2cc3f74a4408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:49.338841 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:49.421017 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb085473-3ce2-49e0-9589-a0090509b352 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:49.430619 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb54d4cb-f486-4d0d-a2c6-86310d711839 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74bb1bc6821b4500b26f160ccae5b7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:49.432024 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb54d4cb-f486-4d0d-a2c6-86310d711839 service nova] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:49.432024 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb54d4cb-f486-4d0d-a2c6-86310d711839 service nova] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:49.432024 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb54d4cb-f486-4d0d-a2c6-86310d711839 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:49.441770 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb54d4cb-f486-4d0d-a2c6-86310d711839 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:49.444025 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb54d4cb-f486-4d0d-a2c6-86310d711839 service nova] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:49.455048 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 851/1722] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:40:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:40:49.463262 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f52ba0-106b-4ada-b371-2cc3f74a4408 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:49.468430 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32bd30d-5ccc-449f-9721-0a4f7ad59442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:49.469315 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32bd30d-5ccc-449f-9721-0a4f7ad59442 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:49.469607 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32bd30d-5ccc-449f-9721-0a4f7ad59442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:49.469940 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32bd30d-5ccc-449f-9721-0a4f7ad59442 admin admin] PATH_INFO: `/v3/projects/b362cc2cf2484182afb8365a5fe5bd98` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:49.471202 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32bd30d-5ccc-449f-9721-0a4f7ad59442 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b362cc2cf2484182afb8365a5fe5bd98)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:49.473284 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32bd30d-5ccc-449f-9721-0a4f7ad59442 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:49.572105 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-e32bd30d-5ccc-449f-9721-0a4f7ad59442 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b362cc2cf2484182afb8365a5fe5bd98', 'request_id': 'req-e32bd30d-5ccc-449f-9721-0a4f7ad59442'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:49.588525 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-e32bd30d-5ccc-449f-9721-0a4f7ad59442 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b362cc2cf2484182afb8365a5fe5bd98', 'request_id': 'req-e32bd30d-5ccc-449f-9721-0a4f7ad59442'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:49.600820 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-e32bd30d-5ccc-449f-9721-0a4f7ad59442 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b362cc2cf2484182afb8365a5fe5bd98', 'request_id': 'req-e32bd30d-5ccc-449f-9721-0a4f7ad59442'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:49.619034 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-e32bd30d-5ccc-449f-9721-0a4f7ad59442 admin admin] The token cache is being invalidate because project b362cc2cf2484182afb8365a5fe5bd98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:40:49.619034 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-e32bd30d-5ccc-449f-9721-0a4f7ad59442 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:49.620185 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-e32bd30d-5ccc-449f-9721-0a4f7ad59442 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:49.622296 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 872/1723] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:40:49 2022] DELETE /identity/v3/projects/b362cc2cf2484182afb8365a5fe5bd98 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:40:49.626640 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb54d4cb-f486-4d0d-a2c6-86310d711839 service nova] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:49.643644 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb54d4cb-f486-4d0d-a2c6-86310d711839 service nova] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:49.719733 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dadfce-20da-4b0b-ad5b-6c319fc2f1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:49.721126 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dadfce-20da-4b0b-ad5b-6c319fc2f1e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:49.721612 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dadfce-20da-4b0b-ad5b-6c319fc2f1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:49.722407 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dadfce-20da-4b0b-ad5b-6c319fc2f1e0 admin admin] PATH_INFO: `/v3/users/2f84b40e57bb4ef98ac50f4f19dca379` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:49.724637 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dadfce-20da-4b0b-ad5b-6c319fc2f1e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f84b40e57bb4ef98ac50f4f19dca379)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:49.727757 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dadfce-20da-4b0b-ad5b-6c319fc2f1e0 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:49.864226 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-46dadfce-20da-4b0b-ad5b-6c319fc2f1e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f84b40e57bb4ef98ac50f4f19dca379'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:49.888997 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-46dadfce-20da-4b0b-ad5b-6c319fc2f1e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f84b40e57bb4ef98ac50f4f19dca379'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:49.889926 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-46dadfce-20da-4b0b-ad5b-6c319fc2f1e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f84b40e57bb4ef98ac50f4f19dca379'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:49.908252 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-46dadfce-20da-4b0b-ad5b-6c319fc2f1e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f84b40e57bb4ef98ac50f4f19dca379'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:49.909154 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-46dadfce-20da-4b0b-ad5b-6c319fc2f1e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f84b40e57bb4ef98ac50f4f19dca379'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:49.934356 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 852/1724] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:40:49 2022] DELETE /identity/v3/users/2f84b40e57bb4ef98ac50f4f19dca379 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:40:50.059675 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32bd30d-5ccc-449f-9721-0a4f7ad59442 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:50.061494 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-e32bd30d-5ccc-449f-9721-0a4f7ad59442 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:50.077316 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32bd30d-5ccc-449f-9721-0a4f7ad59442 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:50.182736 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef12117d-7b45-4a8f-9ab5-0da7ce9688d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:50.182736 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12117d-7b45-4a8f-9ab5-0da7ce9688d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:50.184670 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12117d-7b45-4a8f-9ab5-0da7ce9688d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:50.184670 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12117d-7b45-4a8f-9ab5-0da7ce9688d4 admin admin] PATH_INFO: `/v3/projects/3c9ab5429d4b4a1fb334ed4658bdf701` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:50.185308 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef12117d-7b45-4a8f-9ab5-0da7ce9688d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c9ab5429d4b4a1fb334ed4658bdf701)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:50.187805 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef12117d-7b45-4a8f-9ab5-0da7ce9688d4 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:50.269655 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:50.276485 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-ef12117d-7b45-4a8f-9ab5-0da7ce9688d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c9ab5429d4b4a1fb334ed4658bdf701', 'request_id': 'req-ef12117d-7b45-4a8f-9ab5-0da7ce9688d4'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:50.283941 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-ef12117d-7b45-4a8f-9ab5-0da7ce9688d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c9ab5429d4b4a1fb334ed4658bdf701', 'request_id': 'req-ef12117d-7b45-4a8f-9ab5-0da7ce9688d4'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:50.299871 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-ef12117d-7b45-4a8f-9ab5-0da7ce9688d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c9ab5429d4b4a1fb334ed4658bdf701', 'request_id': 'req-ef12117d-7b45-4a8f-9ab5-0da7ce9688d4'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:50.334728 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-ef12117d-7b45-4a8f-9ab5-0da7ce9688d4 admin admin] The token cache is being invalidate because project 3c9ab5429d4b4a1fb334ed4658bdf701 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:40:50.335575 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-ef12117d-7b45-4a8f-9ab5-0da7ce9688d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:50.337829 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-ef12117d-7b45-4a8f-9ab5-0da7ce9688d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:50.340243 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 873/1725] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:40:50 2022] DELETE /identity/v3/projects/3c9ab5429d4b4a1fb334ed4658bdf701 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:40:50.346284 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb693e-f0f3-4fe3-81bc-ea37f6e3870b None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:50.347464 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb693e-f0f3-4fe3-81bc-ea37f6e3870b None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:50.348120 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb693e-f0f3-4fe3-81bc-ea37f6e3870b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:50.403912 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-dafb693e-f0f3-4fe3-81bc-ea37f6e3870b None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:50.406632 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-dafb693e-f0f3-4fe3-81bc-ea37f6e3870b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:50.437816 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:50.582898 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 853/1726] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:50.588938 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ede9d-3c9b-4769-8957-bfb23b610bdc None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:50.590376 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ede9d-3c9b-4769-8957-bfb23b610bdc None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:50.590766 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ede9d-3c9b-4769-8957-bfb23b610bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:50.630262 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-953ede9d-3c9b-4769-8957-bfb23b610bdc None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:50.631631 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-953ede9d-3c9b-4769-8957-bfb23b610bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:50.657766 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 874/1727] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:50.659902 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb693e-f0f3-4fe3-81bc-ea37f6e3870b None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:50.678773 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb693e-f0f3-4fe3-81bc-ea37f6e3870b None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:50.698329 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9a0fa-ca6a-4868-be74-03b54e299dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:50.701071 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9a0fa-ca6a-4868-be74-03b54e299dd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:50.701490 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9a0fa-ca6a-4868-be74-03b54e299dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:50.702064 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9a0fa-ca6a-4868-be74-03b54e299dd5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:50.706691 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a9a0fa-ca6a-4868-be74-03b54e299dd5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:40:50.709018 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a9a0fa-ca6a-4868-be74-03b54e299dd5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:50.715670 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a9a0fa-ca6a-4868-be74-03b54e299dd5 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:50.759050 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953ede9d-3c9b-4769-8957-bfb23b610bdc None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:50.759185 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 854/1728] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:40:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:40:50.773808 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9a0fa-ca6a-4868-be74-03b54e299dd5 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:50.781013 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-e8a9a0fa-ca6a-4868-be74-03b54e299dd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:50.781940 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953ede9d-3c9b-4769-8957-bfb23b610bdc None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:50.815116 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc6876-11b6-46f1-8f30-70d8d475a4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:50.816012 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc6876-11b6-46f1-8f30-70d8d475a4e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:50.816294 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc6876-11b6-46f1-8f30-70d8d475a4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:50.816578 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc6876-11b6-46f1-8f30-70d8d475a4e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:50.817435 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bc6876-11b6-46f1-8f30-70d8d475a4e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:50.819954 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bc6876-11b6-46f1-8f30-70d8d475a4e1 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:50.843641 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9a0fa-ca6a-4868-be74-03b54e299dd5 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:50.859349 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c9b351-df18-406c-959e-5cd80c6bcd61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:50.860907 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9b351-df18-406c-959e-5cd80c6bcd61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:50.861982 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9b351-df18-406c-959e-5cd80c6bcd61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:50.862862 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9b351-df18-406c-959e-5cd80c6bcd61 admin admin] PATH_INFO: `/v3/projects/c3c13f69c6294c4eab7412b91fc5fd74` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:50.866435 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c9b351-df18-406c-959e-5cd80c6bcd61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3c13f69c6294c4eab7412b91fc5fd74)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:50.871872 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c9b351-df18-406c-959e-5cd80c6bcd61 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:50.886387 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 875/1729] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:40:50 2022] POST /identity/v3/projects => generated 401 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:40:50.890953 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc6876-11b6-46f1-8f30-70d8d475a4e1 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:50.999700 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc6876-11b6-46f1-8f30-70d8d475a4e1 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:51.014771 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-93c9b351-df18-406c-959e-5cd80c6bcd61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3c13f69c6294c4eab7412b91fc5fd74', 'request_id': 'req-93c9b351-df18-406c-959e-5cd80c6bcd61'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:51.033751 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-93c9b351-df18-406c-959e-5cd80c6bcd61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3c13f69c6294c4eab7412b91fc5fd74', 'request_id': 'req-93c9b351-df18-406c-959e-5cd80c6bcd61'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:51.043350 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-93c9b351-df18-406c-959e-5cd80c6bcd61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3c13f69c6294c4eab7412b91fc5fd74', 'request_id': 'req-93c9b351-df18-406c-959e-5cd80c6bcd61'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:51.084569 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-93c9b351-df18-406c-959e-5cd80c6bcd61 admin admin] The token cache is being invalidate because project c3c13f69c6294c4eab7412b91fc5fd74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:40:51.084851 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-93c9b351-df18-406c-959e-5cd80c6bcd61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:51.087799 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-93c9b351-df18-406c-959e-5cd80c6bcd61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:51.094158 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 855/1730] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:40:50 2022] DELETE /identity/v3/projects/c3c13f69c6294c4eab7412b91fc5fd74 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:40:51.100970 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129436a-b101-4e4a-9ad1-9f7717ec7406 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:51.101267 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129436a-b101-4e4a-9ad1-9f7717ec7406 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:51.103829 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129436a-b101-4e4a-9ad1-9f7717ec7406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:51.160145 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-e129436a-b101-4e4a-9ad1-9f7717ec7406 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:51.161774 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-e129436a-b101-4e4a-9ad1-9f7717ec7406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:51.183277 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d2e11-88bf-4056-b7d0-d8090b06bbe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:51.184816 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d2e11-88bf-4056-b7d0-d8090b06bbe6 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:51.185159 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d2e11-88bf-4056-b7d0-d8090b06bbe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:51.186659 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d2e11-88bf-4056-b7d0-d8090b06bbe6 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:51.187435 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1d2e11-88bf-4056-b7d0-d8090b06bbe6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:51.191894 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1d2e11-88bf-4056-b7d0-d8090b06bbe6 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:51.249263 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 876/1731] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:40:50 2022] POST /identity/v3/users => generated 301 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:40:51.256787 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d2e11-88bf-4056-b7d0-d8090b06bbe6 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:51.260366 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-1c1d2e11-88bf-4056-b7d0-d8090b06bbe6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:51.304771 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d2e11-88bf-4056-b7d0-d8090b06bbe6 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:51.319736 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49b2cc0-1632-4404-873e-9356d345cc09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:51.322752 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b2cc0-1632-4404-873e-9356d345cc09 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:51.323299 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b2cc0-1632-4404-873e-9356d345cc09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:51.323826 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b2cc0-1632-4404-873e-9356d345cc09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:51.324760 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49b2cc0-1632-4404-873e-9356d345cc09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:51.327472 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49b2cc0-1632-4404-873e-9356d345cc09 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:51.378206 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 877/1732] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:40:51 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:40:51.380820 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49b2cc0-1632-4404-873e-9356d345cc09 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:51.413921 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49b2cc0-1632-4404-873e-9356d345cc09 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:51.418692 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 856/1733] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:51.428433 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53d9de-c7d0-4d23-ba2e-53b710b0e174 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:51.429682 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53d9de-c7d0-4d23-ba2e-53b710b0e174 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:51.444499 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53d9de-c7d0-4d23-ba2e-53b710b0e174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:51.449561 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8c7cad-124d-4982-a1ad-95fa7b351078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:51.455836 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c7cad-124d-4982-a1ad-95fa7b351078 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:51.456049 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c7cad-124d-4982-a1ad-95fa7b351078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:51.459103 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c7cad-124d-4982-a1ad-95fa7b351078 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:51.461996 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8c7cad-124d-4982-a1ad-95fa7b351078 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:51.468892 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8c7cad-124d-4982-a1ad-95fa7b351078 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:51.566466 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 878/1734] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:40:51 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:40:51.571723 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8c7cad-124d-4982-a1ad-95fa7b351078 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:51.585210 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-4c53d9de-c7d0-4d23-ba2e-53b710b0e174 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:51.586505 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-4c53d9de-c7d0-4d23-ba2e-53b710b0e174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:51.604109 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8c7cad-124d-4982-a1ad-95fa7b351078 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:51.625723 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 857/1735] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:51.626708 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c53d9de-c7d0-4d23-ba2e-53b710b0e174 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:51.631342 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfea8ae-9404-4fa4-8ae8-816fa9575ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:51.637294 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfea8ae-9404-4fa4-8ae8-816fa9575ec2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:51.637784 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfea8ae-9404-4fa4-8ae8-816fa9575ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:51.638238 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfea8ae-9404-4fa4-8ae8-816fa9575ec2 admin admin] PATH_INFO: `/v3/projects/4529288ee9774d0b83e8a2b74567f502/users/f26dfc92f03049c69960194920e13271/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:51.655203 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c53d9de-c7d0-4d23-ba2e-53b710b0e174 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:51.670424 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efb6622-0834-4acf-a3ad-ca032971bb4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:51.671102 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb6622-0834-4acf-a3ad-ca032971bb4f admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:51.671383 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb6622-0834-4acf-a3ad-ca032971bb4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:51.671657 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb6622-0834-4acf-a3ad-ca032971bb4f admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:51.672394 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efb6622-0834-4acf-a3ad-ca032971bb4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:40:51.672695 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efb6622-0834-4acf-a3ad-ca032971bb4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:51.674349 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efb6622-0834-4acf-a3ad-ca032971bb4f admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:51.702590 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfea8ae-9404-4fa4-8ae8-816fa9575ec2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4529288ee9774d0b83e8a2b74567f502, user_id=f26dfc92f03049c69960194920e13271, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:51.704742 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 858/1736] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:40:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:40:51.711374 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efb6622-0834-4acf-a3ad-ca032971bb4f admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:51.721970 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfea8ae-9404-4fa4-8ae8-816fa9575ec2 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:51.725281 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efb6622-0834-4acf-a3ad-ca032971bb4f admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:51.736717 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:DELETE=4,keystone:SELECT=90,keystone:INSERT=6 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:51.740687 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc9a263-894d-4c88-954e-019987267edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:51.741415 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc9a263-894d-4c88-954e-019987267edf admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:51.741720 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc9a263-894d-4c88-954e-019987267edf admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:51.742035 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc9a263-894d-4c88-954e-019987267edf admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:51.742784 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc9a263-894d-4c88-954e-019987267edf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:51.744891 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc9a263-894d-4c88-954e-019987267edf admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:51.767048 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:51.769819 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 879/1737] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:40:51 2022] PUT /identity/v3/projects/4529288ee9774d0b83e8a2b74567f502/users/f26dfc92f03049c69960194920e13271/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:40:51.782882 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa29567-4e73-44bb-927f-6069f1f9fe09 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:51.783380 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa29567-4e73-44bb-927f-6069f1f9fe09 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:51.786147 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa29567-4e73-44bb-927f-6069f1f9fe09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:51.817017 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 859/1738] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:40:51 2022] POST /identity/v3/projects => generated 357 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:40:51.821126 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc9a263-894d-4c88-954e-019987267edf admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:51.847882 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-baa29567-4e73-44bb-927f-6069f1f9fe09 None None] MFA Rules not processed for user `f26dfc92f03049c69960194920e13271`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:51.853213 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-baa29567-4e73-44bb-927f-6069f1f9fe09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:51.973810 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc9a263-894d-4c88-954e-019987267edf admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:51.989846 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c909bd1-2945-42c6-8251-00bc80f342c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:51.992880 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c909bd1-2945-42c6-8251-00bc80f342c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:51.992880 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c909bd1-2945-42c6-8251-00bc80f342c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:51.993119 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c909bd1-2945-42c6-8251-00bc80f342c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:51.996131 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c909bd1-2945-42c6-8251-00bc80f342c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:51.999888 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c909bd1-2945-42c6-8251-00bc80f342c3 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:52.021863 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 880/1739] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:51 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:52.023298 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e7af2-9eaf-4809-9216-3fe391a1f501 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:52.023589 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e7af2-9eaf-4809-9216-3fe391a1f501 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:52.026781 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e7af2-9eaf-4809-9216-3fe391a1f501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:52.063391 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 860/1740] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:40:51 2022] POST /identity/v3/users => generated 279 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:40:52.068420 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c909bd1-2945-42c6-8251-00bc80f342c3 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:52.082431 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-e29e7af2-9eaf-4809-9216-3fe391a1f501 None None] MFA Rules not processed for user `75a612089b2b46a6b3ff5a6cd9e59579`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:52.084748 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-e29e7af2-9eaf-4809-9216-3fe391a1f501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:52.098305 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c909bd1-2945-42c6-8251-00bc80f342c3 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:52.119945 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e78fff-3244-48f8-a582-1a33af87365e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:52.123254 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e78fff-3244-48f8-a582-1a33af87365e admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:52.123605 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e78fff-3244-48f8-a582-1a33af87365e admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:52.123836 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e78fff-3244-48f8-a582-1a33af87365e admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:52.127762 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e78fff-3244-48f8-a582-1a33af87365e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:52.136042 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e78fff-3244-48f8-a582-1a33af87365e admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:52.208028 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 861/1741] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:40:52 2022] POST /identity/v3/projects => generated 401 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:40:52.210779 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e78fff-3244-48f8-a582-1a33af87365e admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:52.277740 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 881/1742] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:52.278672 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29e7af2-9eaf-4809-9216-3fe391a1f501 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:52.300250 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29e7af2-9eaf-4809-9216-3fe391a1f501 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:52.378587 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e78fff-3244-48f8-a582-1a33af87365e admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:52.395033 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd28ad8-aeea-4761-bf83-9a3ae73f23f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:52.395770 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd28ad8-aeea-4761-bf83-9a3ae73f23f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:52.396662 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd28ad8-aeea-4761-bf83-9a3ae73f23f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:52.399346 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd28ad8-aeea-4761-bf83-9a3ae73f23f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:52.400183 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd28ad8-aeea-4761-bf83-9a3ae73f23f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:52.402318 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd28ad8-aeea-4761-bf83-9a3ae73f23f7 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:52.421540 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 862/1743] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:40:52 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:40:52.425902 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd28ad8-aeea-4761-bf83-9a3ae73f23f7 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:52.445225 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd28ad8-aeea-4761-bf83-9a3ae73f23f7 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:52.451340 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad32a15-71f9-4ee3-b8ee-41f29d88b662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:52.452243 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad32a15-71f9-4ee3-b8ee-41f29d88b662 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:52.452684 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad32a15-71f9-4ee3-b8ee-41f29d88b662 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:52.453127 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad32a15-71f9-4ee3-b8ee-41f29d88b662 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:52.454124 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad32a15-71f9-4ee3-b8ee-41f29d88b662 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:52.456692 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad32a15-71f9-4ee3-b8ee-41f29d88b662 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:52.470930 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 863/1744] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:40:52 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:40:52.476258 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad32a15-71f9-4ee3-b8ee-41f29d88b662 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:52.523024 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad32a15-71f9-4ee3-b8ee-41f29d88b662 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:52.531738 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734941b7-68a3-4330-a6c6-4662d0dc7e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:52.532753 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734941b7-68a3-4330-a6c6-4662d0dc7e7c admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:52.533167 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734941b7-68a3-4330-a6c6-4662d0dc7e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:52.533660 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734941b7-68a3-4330-a6c6-4662d0dc7e7c admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:52.534824 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734941b7-68a3-4330-a6c6-4662d0dc7e7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:52.538064 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734941b7-68a3-4330-a6c6-4662d0dc7e7c admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:52.539829 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f2a32-2aac-4e65-9c2d-8e98515de2fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:52.540824 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f2a32-2aac-4e65-9c2d-8e98515de2fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:52.541113 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f2a32-2aac-4e65-9c2d-8e98515de2fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:52.541392 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f2a32-2aac-4e65-9c2d-8e98515de2fb admin admin] PATH_INFO: `/v3/projects/e6b2d4882c634c8f813aa423ab719fe5/users/872badbeaf524e6d873814e961f35f3e/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:52.574418 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 882/1745] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:40:52 2022] POST /identity/v3/users => generated 307 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:40:52.577398 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734941b7-68a3-4330-a6c6-4662d0dc7e7c admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:52.586649 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325f2a32-2aac-4e65-9c2d-8e98515de2fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6b2d4882c634c8f813aa423ab719fe5, user_id=872badbeaf524e6d873814e961f35f3e, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:52.592159 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325f2a32-2aac-4e65-9c2d-8e98515de2fb admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:52.604611 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734941b7-68a3-4330-a6c6-4662d0dc7e7c admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:52.621528 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6f74bb-87cb-4ff4-8620-7082c9ce5020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:52.623975 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f74bb-87cb-4ff4-8620-7082c9ce5020 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:52.624649 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f74bb-87cb-4ff4-8620-7082c9ce5020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:52.625250 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f74bb-87cb-4ff4-8620-7082c9ce5020 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:52.626042 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6f74bb-87cb-4ff4-8620-7082c9ce5020 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:52.631749 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6f74bb-87cb-4ff4-8620-7082c9ce5020 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:52.643580 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 864/1746] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:40:52 2022] PUT /identity/v3/projects/e6b2d4882c634c8f813aa423ab719fe5/users/872badbeaf524e6d873814e961f35f3e/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:40:52.658150 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161146fd-c063-40fd-959b-0b8922fdd553 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:52.658772 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161146fd-c063-40fd-959b-0b8922fdd553 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:52.659222 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161146fd-c063-40fd-959b-0b8922fdd553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:52.666084 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 883/1747] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:40:52 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:40:52.667072 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6f74bb-87cb-4ff4-8620-7082c9ce5020 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:52.691866 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6f74bb-87cb-4ff4-8620-7082c9ce5020 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:52.713195 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:52.730451 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-161146fd-c063-40fd-959b-0b8922fdd553 None None] MFA Rules not processed for user `872badbeaf524e6d873814e961f35f3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:52.731064 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-161146fd-c063-40fd-959b-0b8922fdd553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:52.740045 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:52.837015 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f11b19-7498-49f0-b06e-f2cbc2074db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:52.839033 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f11b19-7498-49f0-b06e-f2cbc2074db7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:52.839603 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f11b19-7498-49f0-b06e-f2cbc2074db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:52.840075 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f11b19-7498-49f0-b06e-f2cbc2074db7 admin admin] PATH_INFO: `/v3/projects/70497f6317a24cd18bb12d6e9779064c/users/bad78bfdc12045a482c864908001b4a2/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:52.906481 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 865/1748] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:52 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:40:52.907183 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f11b19-7498-49f0-b06e-f2cbc2074db7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70497f6317a24cd18bb12d6e9779064c, user_id=bad78bfdc12045a482c864908001b4a2, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:52.910152 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f11b19-7498-49f0-b06e-f2cbc2074db7 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:52.913466 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161146fd-c063-40fd-959b-0b8922fdd553 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:52.936968 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161146fd-c063-40fd-959b-0b8922fdd553 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:52.949620 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 884/1749] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:40:52 2022] PUT /identity/v3/projects/70497f6317a24cd18bb12d6e9779064c/users/bad78bfdc12045a482c864908001b4a2/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:40:52.954527 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f11b19-7498-49f0-b06e-f2cbc2074db7 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:53.102068 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b306b7a6-a1f6-4ae4-b616-7ebe894d50ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:53.103622 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b306b7a6-a1f6-4ae4-b616-7ebe894d50ad admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:53.104079 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b306b7a6-a1f6-4ae4-b616-7ebe894d50ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:53.104503 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b306b7a6-a1f6-4ae4-b616-7ebe894d50ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:53.105377 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b306b7a6-a1f6-4ae4-b616-7ebe894d50ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:53.107983 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b306b7a6-a1f6-4ae4-b616-7ebe894d50ad admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:53.145340 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f11b19-7498-49f0-b06e-f2cbc2074db7 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:53.154761 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9106ef62-bdc2-4751-a18c-4f9263848232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:53.155922 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106ef62-bdc2-4751-a18c-4f9263848232 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:53.156130 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106ef62-bdc2-4751-a18c-4f9263848232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:53.156416 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106ef62-bdc2-4751-a18c-4f9263848232 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:53.157167 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9106ef62-bdc2-4751-a18c-4f9263848232 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:53.158828 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9106ef62-bdc2-4751-a18c-4f9263848232 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:53.196473 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 885/1750] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:40:52 2022] GET /identity/v3/roles => generated 1420 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:40:53.197170 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9106ef62-bdc2-4751-a18c-4f9263848232 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:53.198926 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-9106ef62-bdc2-4751-a18c-4f9263848232 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:53.205177 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 866/1751] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:40:52 2022] POST /identity/v3/projects => generated 355 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:40:53.207425 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ce4e2-211a-4fd9-802b-05f86ab46b02 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:53.209316 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ce4e2-211a-4fd9-802b-05f86ab46b02 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:53.209896 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ce4e2-211a-4fd9-802b-05f86ab46b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:53.224976 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9106ef62-bdc2-4751-a18c-4f9263848232 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:53.340271 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-772ce4e2-211a-4fd9-802b-05f86ab46b02 None None] MFA Rules not processed for user `c8215c41d0c54c62b4c9fea630481000`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:53.341537 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-772ce4e2-211a-4fd9-802b-05f86ab46b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:53.366377 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2231c9-ab21-4fb0-9b07-b4285497eea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:53.371592 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2231c9-ab21-4fb0-9b07-b4285497eea9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:53.371592 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2231c9-ab21-4fb0-9b07-b4285497eea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:53.371592 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2231c9-ab21-4fb0-9b07-b4285497eea9 admin admin] PATH_INFO: `/v3/users/53a314460e6b4a1b9b9a78059e8bcd9e` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:53.371592 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2231c9-ab21-4fb0-9b07-b4285497eea9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53a314460e6b4a1b9b9a78059e8bcd9e)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:53.371592 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2231c9-ab21-4fb0-9b07-b4285497eea9 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:53.453006 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:53.480218 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-8c2231c9-ab21-4fb0-9b07-b4285497eea9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53a314460e6b4a1b9b9a78059e8bcd9e'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:53.507030 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-8c2231c9-ab21-4fb0-9b07-b4285497eea9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53a314460e6b4a1b9b9a78059e8bcd9e'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:53.535058 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-8c2231c9-ab21-4fb0-9b07-b4285497eea9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53a314460e6b4a1b9b9a78059e8bcd9e'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:53.535770 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-8c2231c9-ab21-4fb0-9b07-b4285497eea9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53a314460e6b4a1b9b9a78059e8bcd9e'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:53.536960 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-8c2231c9-ab21-4fb0-9b07-b4285497eea9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53a314460e6b4a1b9b9a78059e8bcd9e'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:53.557337 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 886/1752] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:40:53 2022] DELETE /identity/v3/users/53a314460e6b4a1b9b9a78059e8bcd9e => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:40:53.561309 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2231c9-ab21-4fb0-9b07-b4285497eea9 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:53.562869 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-8c2231c9-ab21-4fb0-9b07-b4285497eea9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:53.607286 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 867/1753] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 401 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:53.608214 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772ce4e2-211a-4fd9-802b-05f86ab46b02 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:53.609647 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-772ce4e2-211a-4fd9-802b-05f86ab46b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:53.637496 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772ce4e2-211a-4fd9-802b-05f86ab46b02 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:53.651248 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bf2291-4883-4cec-974f-8a04827789fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:53.651983 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf2291-4883-4cec-974f-8a04827789fe admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:53.652265 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf2291-4883-4cec-974f-8a04827789fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:53.652546 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf2291-4883-4cec-974f-8a04827789fe admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:53.653311 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bf2291-4883-4cec-974f-8a04827789fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:53.655372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bf2291-4883-4cec-974f-8a04827789fe admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:53.664946 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2231c9-ab21-4fb0-9b07-b4285497eea9 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:53.675296 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0788ee3-9f7d-4b7a-bf3f-ad18d07e16ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:53.676969 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0788ee3-9f7d-4b7a-bf3f-ad18d07e16ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:53.679250 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0788ee3-9f7d-4b7a-bf3f-ad18d07e16ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:53.680406 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0788ee3-9f7d-4b7a-bf3f-ad18d07e16ef admin admin] PATH_INFO: `/v3/projects/70497f6317a24cd18bb12d6e9779064c/users/bad78bfdc12045a482c864908001b4a2/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:53.687347 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0788ee3-9f7d-4b7a-bf3f-ad18d07e16ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=70497f6317a24cd18bb12d6e9779064c, user_id=bad78bfdc12045a482c864908001b4a2, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:53.691862 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0788ee3-9f7d-4b7a-bf3f-ad18d07e16ef admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:53.711316 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 887/1754] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:40:53 2022] PUT /identity/v3/projects/70497f6317a24cd18bb12d6e9779064c/users/bad78bfdc12045a482c864908001b4a2/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:40:53.714515 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695cb59c-f1c5-41e5-9fcf-ac27b4e091d2 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:53.714991 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695cb59c-f1c5-41e5-9fcf-ac27b4e091d2 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:53.715332 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 868/1755] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:40:53 2022] POST /identity/v3/users => generated 284 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:40:53.715574 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695cb59c-f1c5-41e5-9fcf-ac27b4e091d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:53.716946 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bfbe9d-739e-4b24-b1f1-ddc3b2b679b0 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:53.717445 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bfbe9d-739e-4b24-b1f1-ddc3b2b679b0 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:53.717881 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bfbe9d-739e-4b24-b1f1-ddc3b2b679b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:53.730747 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:53.758628 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-695cb59c-f1c5-41e5-9fcf-ac27b4e091d2 None None] MFA Rules not processed for user `71e0fd73e5c9430c813af1de9ccd4a52`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:53.761123 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-695cb59c-f1c5-41e5-9fcf-ac27b4e091d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:53.765108 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-b1bfbe9d-739e-4b24-b1f1-ddc3b2b679b0 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:53.768954 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-b1bfbe9d-739e-4b24-b1f1-ddc3b2b679b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:53.891546 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 888/1756] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:53.893118 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee5b475-75e2-4941-98f6-d5818d22f5e5 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:53.893520 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee5b475-75e2-4941-98f6-d5818d22f5e5 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:53.894198 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee5b475-75e2-4941-98f6-d5818d22f5e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:53.909127 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 869/1757] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:53.920151 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bfbe9d-739e-4b24-b1f1-ddc3b2b679b0 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:53.940411 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-9ee5b475-75e2-4941-98f6-d5818d22f5e5 None None] MFA Rules not processed for user `c8215c41d0c54c62b4c9fea630481000`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:53.942133 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-9ee5b475-75e2-4941-98f6-d5818d22f5e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:53.946511 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bfbe9d-739e-4b24-b1f1-ddc3b2b679b0 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:53.961023 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4667bcd5-fdfd-4d64-b574-6ae9cae7e55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:53.961912 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667bcd5-fdfd-4d64-b574-6ae9cae7e55c admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:53.962306 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667bcd5-fdfd-4d64-b574-6ae9cae7e55c admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:53.962696 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667bcd5-fdfd-4d64-b574-6ae9cae7e55c admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:53.963522 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4667bcd5-fdfd-4d64-b574-6ae9cae7e55c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:53.965368 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4667bcd5-fdfd-4d64-b574-6ae9cae7e55c admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:53.994503 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:54.000838 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 870/1758] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:40:53 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:40:54.002155 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e971f1b9-9b70-4f8a-89e6-3706f394cb68 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:54.003919 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e971f1b9-9b70-4f8a-89e6-3706f394cb68 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:54.004293 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e971f1b9-9b70-4f8a-89e6-3706f394cb68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:54.043689 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 889/1759] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:54.045530 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183798d3-81fd-4aeb-9e16-0e782af57037 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:54.046006 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183798d3-81fd-4aeb-9e16-0e782af57037 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:54.046427 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183798d3-81fd-4aeb-9e16-0e782af57037 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:54.055692 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 890/1760] 10.222.0.13 () {54 vars in 912 bytes} [Fri Jun 24 08:40:54 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:40:54.057381 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908e4cd-f3e3-4454-a540-00c91554d29c None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:54.057683 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908e4cd-f3e3-4454-a540-00c91554d29c None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:54.057987 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908e4cd-f3e3-4454-a540-00c91554d29c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:54.072797 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-e971f1b9-9b70-4f8a-89e6-3706f394cb68 None None] MFA Rules not processed for user `bad78bfdc12045a482c864908001b4a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:54.074166 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-e971f1b9-9b70-4f8a-89e6-3706f394cb68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:54.080695 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-1908e4cd-f3e3-4454-a540-00c91554d29c None None] MFA Rules not processed for user `71e0fd73e5c9430c813af1de9ccd4a52`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:54.082222 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-1908e4cd-f3e3-4454-a540-00c91554d29c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:54.092531 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 891/1761] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:54.097629 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908e4cd-f3e3-4454-a540-00c91554d29c None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:54.099637 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-1908e4cd-f3e3-4454-a540-00c91554d29c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:54.177516 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908e4cd-f3e3-4454-a540-00c91554d29c None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:54.183367 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c83c55-06b6-4f0e-ae74-d9d6bf29f764 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:54.183899 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c83c55-06b6-4f0e-ae74-d9d6bf29f764 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:54.184186 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c83c55-06b6-4f0e-ae74-d9d6bf29f764 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:54.184523 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c83c55-06b6-4f0e-ae74-d9d6bf29f764 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:54.201260 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c83c55-06b6-4f0e-ae74-d9d6bf29f764 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:54.203224 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c83c55-06b6-4f0e-ae74-d9d6bf29f764 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:54.219453 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 892/1762] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:40:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:40:54.220542 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c83c55-06b6-4f0e-ae74-d9d6bf29f764 service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:54.236496 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c83c55-06b6-4f0e-ae74-d9d6bf29f764 service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:54.238273 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 871/1763] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:53 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:54.239387 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e971f1b9-9b70-4f8a-89e6-3706f394cb68 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:54.241007 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-e971f1b9-9b70-4f8a-89e6-3706f394cb68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:54.257063 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3580a24c-872f-4b43-9b3d-54b99b5b5a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:54.258517 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580a24c-872f-4b43-9b3d-54b99b5b5a95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:54.259109 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580a24c-872f-4b43-9b3d-54b99b5b5a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:54.259574 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580a24c-872f-4b43-9b3d-54b99b5b5a95 admin admin] PATH_INFO: `/v3/projects/1ef8bdc250be4dbfabceb417a73bc2ec/users/359474792f2548a2b2b73c88898d9abb/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:54.289870 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3580a24c-872f-4b43-9b3d-54b99b5b5a95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ef8bdc250be4dbfabceb417a73bc2ec, user_id=359474792f2548a2b2b73c88898d9abb, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:54.301825 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3580a24c-872f-4b43-9b3d-54b99b5b5a95 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:54.395084 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 893/1764] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:40:54 2022] PUT /identity/v3/projects/1ef8bdc250be4dbfabceb417a73bc2ec/users/359474792f2548a2b2b73c88898d9abb/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:40:54.411629 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a56b5a-95ac-4652-a660-fca74828ce95 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:54.412244 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a56b5a-95ac-4652-a660-fca74828ce95 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:54.412610 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a56b5a-95ac-4652-a660-fca74828ce95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:54.471861 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-c3a56b5a-95ac-4652-a660-fca74828ce95 None None] MFA Rules not processed for user `c5f7ba6c7faf43e7acbb43eee58d8c8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:54.473403 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-c3a56b5a-95ac-4652-a660-fca74828ce95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:54.536687 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e971f1b9-9b70-4f8a-89e6-3706f394cb68 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:54.558383 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:54.639851 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 894/1765] 10.222.0.13 () {58 vars in 1009 bytes} [Fri Jun 24 08:40:54 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:54.642138 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a56b5a-95ac-4652-a660-fca74828ce95 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:54.643823 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-c3a56b5a-95ac-4652-a660-fca74828ce95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:54.699515 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd8b6e6-c321-4805-a343-140287a1059d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74bb1bc6821b4500b26f160ccae5b7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:54.700199 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd8b6e6-c321-4805-a343-140287a1059d service nova] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:54.700468 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd8b6e6-c321-4805-a343-140287a1059d service nova] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:54.700957 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd8b6e6-c321-4805-a343-140287a1059d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:54.747333 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:54.777791 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a56b5a-95ac-4652-a660-fca74828ce95 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:54.787936 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10da9cdc-6330-4c23-b246-92f4f47ae23c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:54.789081 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da9cdc-6330-4c23-b246-92f4f47ae23c service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:54.789474 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da9cdc-6330-4c23-b246-92f4f47ae23c service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:54.789849 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da9cdc-6330-4c23-b246-92f4f47ae23c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:54.894162 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd8b6e6-c321-4805-a343-140287a1059d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:54.906980 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10da9cdc-6330-4c23-b246-92f4f47ae23c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:54.908456 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10da9cdc-6330-4c23-b246-92f4f47ae23c service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:54.916389 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd8b6e6-c321-4805-a343-140287a1059d service nova] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:54.936810 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 895/1766] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:40:54 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:40:54.938314 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36ab162-2304-4f3e-a3d8-6bef40647f82 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:54.938606 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36ab162-2304-4f3e-a3d8-6bef40647f82 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:54.938885 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36ab162-2304-4f3e-a3d8-6bef40647f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:54.946692 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 872/1767] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:40:54 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 707 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:40:54.948774 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd8b6e6-c321-4805-a343-140287a1059d service nova] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:55.004437 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-e36ab162-2304-4f3e-a3d8-6bef40647f82 None None] MFA Rules not processed for user `71cbad05d1a54b4b9959e2ebbcb720fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:55.006654 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-e36ab162-2304-4f3e-a3d8-6bef40647f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:55.107066 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd8b6e6-c321-4805-a343-140287a1059d service nova] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:55.112342 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6279203-52c8-4662-b23f-adbb271e8ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:55.113207 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6279203-52c8-4662-b23f-adbb271e8ed9 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:55.114122 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6279203-52c8-4662-b23f-adbb271e8ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:55.114122 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6279203-52c8-4662-b23f-adbb271e8ed9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:55.116127 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6279203-52c8-4662-b23f-adbb271e8ed9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:55.118865 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6279203-52c8-4662-b23f-adbb271e8ed9 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:55.137075 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 873/1768] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:40:54 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:40:55.138690 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6279203-52c8-4662-b23f-adbb271e8ed9 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:55.141205 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-c6279203-52c8-4662-b23f-adbb271e8ed9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:55.172441 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6279203-52c8-4662-b23f-adbb271e8ed9 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:55.177322 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d8a84-f1a7-4d08-b013-318daca5411c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:55.178239 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d8a84-f1a7-4d08-b013-318daca5411c service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:55.178504 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d8a84-f1a7-4d08-b013-318daca5411c service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:55.178787 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d8a84-f1a7-4d08-b013-318daca5411c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:55.184454 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 896/1769] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:55.184454 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36ab162-2304-4f3e-a3d8-6bef40647f82 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:55.185676 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-e36ab162-2304-4f3e-a3d8-6bef40647f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:55.189717 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52d8a84-f1a7-4d08-b013-318daca5411c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:55.191643 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52d8a84-f1a7-4d08-b013-318daca5411c service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:55.203752 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 874/1770] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:40:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:40:55.205560 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d8a84-f1a7-4d08-b013-318daca5411c service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:55.207538 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-c52d8a84-f1a7-4d08-b013-318daca5411c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:55.229242 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36ab162-2304-4f3e-a3d8-6bef40647f82 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:55.247361 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15685bb7-2d15-422a-afd0-df501a3b7dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:55.248646 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15685bb7-2d15-422a-afd0-df501a3b7dd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:55.248926 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15685bb7-2d15-422a-afd0-df501a3b7dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:55.249295 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15685bb7-2d15-422a-afd0-df501a3b7dd6 admin admin] PATH_INFO: `/v3/projects/2ab3fbea9d924850b2d282c31c874aae` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:55.251741 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15685bb7-2d15-422a-afd0-df501a3b7dd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ab3fbea9d924850b2d282c31c874aae)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:55.253837 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15685bb7-2d15-422a-afd0-df501a3b7dd6 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:55.276353 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d8a84-f1a7-4d08-b013-318daca5411c service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:55.307720 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88db112-b00b-4767-8fc3-3e5f614daa5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:55.308830 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88db112-b00b-4767-8fc3-3e5f614daa5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:55.309115 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88db112-b00b-4767-8fc3-3e5f614daa5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:55.309390 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88db112-b00b-4767-8fc3-3e5f614daa5c admin admin] PATH_INFO: `/v3/users/d333ec005de042a9b8fc8cdc78c96ee8` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:55.310749 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88db112-b00b-4767-8fc3-3e5f614daa5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d333ec005de042a9b8fc8cdc78c96ee8)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:55.314912 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88db112-b00b-4767-8fc3-3e5f614daa5c admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:55.415913 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:55.445978 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-15685bb7-2d15-422a-afd0-df501a3b7dd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ab3fbea9d924850b2d282c31c874aae', 'request_id': 'req-15685bb7-2d15-422a-afd0-df501a3b7dd6'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:55.457664 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-15685bb7-2d15-422a-afd0-df501a3b7dd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ab3fbea9d924850b2d282c31c874aae', 'request_id': 'req-15685bb7-2d15-422a-afd0-df501a3b7dd6'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:55.460311 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-b88db112-b00b-4767-8fc3-3e5f614daa5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd333ec005de042a9b8fc8cdc78c96ee8'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:55.470678 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-15685bb7-2d15-422a-afd0-df501a3b7dd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ab3fbea9d924850b2d282c31c874aae', 'request_id': 'req-15685bb7-2d15-422a-afd0-df501a3b7dd6'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:55.481503 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-b88db112-b00b-4767-8fc3-3e5f614daa5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd333ec005de042a9b8fc8cdc78c96ee8'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:55.483055 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-b88db112-b00b-4767-8fc3-3e5f614daa5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd333ec005de042a9b8fc8cdc78c96ee8'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:55.499053 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-15685bb7-2d15-422a-afd0-df501a3b7dd6 admin admin] The token cache is being invalidate because project 2ab3fbea9d924850b2d282c31c874aae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:40:55.499238 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-15685bb7-2d15-422a-afd0-df501a3b7dd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:55.501444 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-15685bb7-2d15-422a-afd0-df501a3b7dd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:55.506114 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 897/1771] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:40:55 2022] DELETE /identity/v3/projects/2ab3fbea9d924850b2d282c31c874aae => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:40:55.506767 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15685bb7-2d15-422a-afd0-df501a3b7dd6 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:55.508425 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-b88db112-b00b-4767-8fc3-3e5f614daa5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd333ec005de042a9b8fc8cdc78c96ee8'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:55.509286 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-b88db112-b00b-4767-8fc3-3e5f614daa5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd333ec005de042a9b8fc8cdc78c96ee8'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:55.510546 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-15685bb7-2d15-422a-afd0-df501a3b7dd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:55.535550 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 875/1772] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:40:55 2022] DELETE /identity/v3/users/d333ec005de042a9b8fc8cdc78c96ee8 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:40:55.536110 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88db112-b00b-4767-8fc3-3e5f614daa5c admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:55.539650 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-b88db112-b00b-4767-8fc3-3e5f614daa5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:55.571615 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:55.643329 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15685bb7-2d15-422a-afd0-df501a3b7dd6 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:55.687180 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88db112-b00b-4767-8fc3-3e5f614daa5c admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:55.705526 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71c189d-b8b0-4710-ac67-28b73faf6961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:55.706737 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c189d-b8b0-4710-ac67-28b73faf6961 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:55.707023 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c189d-b8b0-4710-ac67-28b73faf6961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:55.707297 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c189d-b8b0-4710-ac67-28b73faf6961 admin admin] PATH_INFO: `/v3/projects/1ef8bdc250be4dbfabceb417a73bc2ec/users/359474792f2548a2b2b73c88898d9abb/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:55.711744 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71c189d-b8b0-4710-ac67-28b73faf6961 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ef8bdc250be4dbfabceb417a73bc2ec, user_id=359474792f2548a2b2b73c88898d9abb, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:55.714705 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71c189d-b8b0-4710-ac67-28b73faf6961 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:55.747821 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 876/1773] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:40:55 2022] PUT /identity/v3/projects/1ef8bdc250be4dbfabceb417a73bc2ec/users/359474792f2548a2b2b73c88898d9abb/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:40:55.750878 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71c189d-b8b0-4710-ac67-28b73faf6961 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:55.760131 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-c71c189d-b8b0-4710-ac67-28b73faf6961 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:55.772157 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6939474f-e5d4-441e-8d85-25c16a3e760e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:55.772842 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939474f-e5d4-441e-8d85-25c16a3e760e service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:55.773126 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939474f-e5d4-441e-8d85-25c16a3e760e service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:55.773391 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939474f-e5d4-441e-8d85-25c16a3e760e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:55.780585 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-6939474f-e5d4-441e-8d85-25c16a3e760e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:55.949222 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6939474f-e5d4-441e-8d85-25c16a3e760e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:55.950907 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6939474f-e5d4-441e-8d85-25c16a3e760e service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:55.953868 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71c189d-b8b0-4710-ac67-28b73faf6961 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:55.961211 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0163fa-9334-4c5f-bcf3-08fee0640855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:55.962301 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0163fa-9334-4c5f-bcf3-08fee0640855 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:55.962803 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0163fa-9334-4c5f-bcf3-08fee0640855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:55.963023 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0163fa-9334-4c5f-bcf3-08fee0640855 admin admin] PATH_INFO: `/v3/users/b5db2014719146c5a014d97a334c78b6` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:55.964564 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0163fa-9334-4c5f-bcf3-08fee0640855 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5db2014719146c5a014d97a334c78b6)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:55.965176 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 898/1774] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:40:55 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:40:55.966598 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200fd1bc-9181-45b4-9648-fdc9b881c37f None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:55.966952 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200fd1bc-9181-45b4-9648-fdc9b881c37f None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:55.967304 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200fd1bc-9181-45b4-9648-fdc9b881c37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:55.967853 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0163fa-9334-4c5f-bcf3-08fee0640855 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:55.991593 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-200fd1bc-9181-45b4-9648-fdc9b881c37f None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:55.994340 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-200fd1bc-9181-45b4-9648-fdc9b881c37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:56.015232 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 899/1775] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:56.016855 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49226f2d-e176-4023-8c56-701c47415d1d None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:56.017241 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49226f2d-e176-4023-8c56-701c47415d1d None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:56.017588 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49226f2d-e176-4023-8c56-701c47415d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:56.055986 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-49226f2d-e176-4023-8c56-701c47415d1d None None] MFA Rules not processed for user `359474792f2548a2b2b73c88898d9abb`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:56.057505 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-49226f2d-e176-4023-8c56-701c47415d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:56.095536 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:56.099959 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-fe0163fa-9334-4c5f-bcf3-08fee0640855 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5db2014719146c5a014d97a334c78b6'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:56.139490 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-fe0163fa-9334-4c5f-bcf3-08fee0640855 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5db2014719146c5a014d97a334c78b6'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:56.145575 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-fe0163fa-9334-4c5f-bcf3-08fee0640855 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5db2014719146c5a014d97a334c78b6'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:56.166728 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-fe0163fa-9334-4c5f-bcf3-08fee0640855 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5db2014719146c5a014d97a334c78b6'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:56.168254 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-fe0163fa-9334-4c5f-bcf3-08fee0640855 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5db2014719146c5a014d97a334c78b6'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:56.205113 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 877/1776] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:40:55 2022] DELETE /identity/v3/users/b5db2014719146c5a014d97a334c78b6 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:40:56.206070 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0163fa-9334-4c5f-bcf3-08fee0640855 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:56.211491 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-fe0163fa-9334-4c5f-bcf3-08fee0640855 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:56.216618 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 900/1777] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:56 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:56.218812 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a332ef55-62ad-4634-90ae-f145426ad541 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:56.219011 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a332ef55-62ad-4634-90ae-f145426ad541 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:56.219251 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a332ef55-62ad-4634-90ae-f145426ad541 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:56.220619 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 901/1778] 10.222.0.13 () {54 vars in 912 bytes} [Fri Jun 24 08:40:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:40:56.221363 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a332ef55-62ad-4634-90ae-f145426ad541 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:56.236660 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-a332ef55-62ad-4634-90ae-f145426ad541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:56.290519 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:56.479918 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0163fa-9334-4c5f-bcf3-08fee0640855 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:56.503716 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a812d0b6-338a-4f0b-9a29-548beeca1bf3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74bb1bc6821b4500b26f160ccae5b7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:56.512274 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812d0b6-338a-4f0b-9a29-548beeca1bf3 service nova] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:56.512935 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812d0b6-338a-4f0b-9a29-548beeca1bf3 service nova] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:56.513378 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812d0b6-338a-4f0b-9a29-548beeca1bf3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:56.518379 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-a812d0b6-338a-4f0b-9a29-548beeca1bf3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:56.527556 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a332ef55-62ad-4634-90ae-f145426ad541 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:56.546684 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e61920f-c221-48b5-ab06-c481f927bf23 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39cc268bad604dd08594e3b39c7ecf5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:56.547842 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61920f-c221-48b5-ab06-c481f927bf23 service placement] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:56.547842 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61920f-c221-48b5-ab06-c481f927bf23 service placement] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:56.548092 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61920f-c221-48b5-ab06-c481f927bf23 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:56.559638 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-0e61920f-c221-48b5-ab06-c481f927bf23 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:56.624250 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e61920f-c221-48b5-ab06-c481f927bf23 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:56.634332 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e61920f-c221-48b5-ab06-c481f927bf23 service placement] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:56.673876 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 902/1779] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:40:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:40:56.676501 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e61920f-c221-48b5-ab06-c481f927bf23 service placement] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:56.687414 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-0e61920f-c221-48b5-ab06-c481f927bf23 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:56.703555 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a812d0b6-338a-4f0b-9a29-548beeca1bf3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:56.706644 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a812d0b6-338a-4f0b-9a29-548beeca1bf3 service nova] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:56.720560 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 878/1780] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:40:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:40:56.722181 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3907332-c1cd-48bb-b9e9-059cb9cb52fc None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:56.722572 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3907332-c1cd-48bb-b9e9-059cb9cb52fc None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:56.723029 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3907332-c1cd-48bb-b9e9-059cb9cb52fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:56.838948 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-e3907332-c1cd-48bb-b9e9-059cb9cb52fc None None] MFA Rules not processed for user `c5f7ba6c7faf43e7acbb43eee58d8c8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:56.845962 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-e3907332-c1cd-48bb-b9e9-059cb9cb52fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:56.916787 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e61920f-c221-48b5-ab06-c481f927bf23 service placement] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:56.935491 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034efaa1-c1f9-4ec5-865f-380f07446bab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:56.936174 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034efaa1-c1f9-4ec5-865f-380f07446bab service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:56.936474 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034efaa1-c1f9-4ec5-865f-380f07446bab service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:56.936735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034efaa1-c1f9-4ec5-865f-380f07446bab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:56.944403 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-034efaa1-c1f9-4ec5-865f-380f07446bab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:56.994032 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 879/1781] 10.222.0.13 () {58 vars in 1009 bytes} [Fri Jun 24 08:40:56 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:56.995542 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3907332-c1cd-48bb-b9e9-059cb9cb52fc None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:56.998012 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-e3907332-c1cd-48bb-b9e9-059cb9cb52fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:57.010090 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:57.116167 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034efaa1-c1f9-4ec5-865f-380f07446bab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:57.117988 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034efaa1-c1f9-4ec5-865f-380f07446bab service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:57.137098 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 903/1782] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:40:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:40:57.137933 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034efaa1-c1f9-4ec5-865f-380f07446bab service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:57.152342 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034efaa1-c1f9-4ec5-865f-380f07446bab service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:57.156483 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7247a4b-3f50-4c31-9c21-010529ddd83f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:57.156940 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3907332-c1cd-48bb-b9e9-059cb9cb52fc None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:57.157236 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7247a4b-3f50-4c31-9c21-010529ddd83f service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:57.157521 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7247a4b-3f50-4c31-9c21-010529ddd83f service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:57.157827 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7247a4b-3f50-4c31-9c21-010529ddd83f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:57.166185 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9b716b-35ae-486c-ac74-57fd87c09001 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:57.167331 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b716b-35ae-486c-ac74-57fd87c09001 service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:57.168223 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b716b-35ae-486c-ac74-57fd87c09001 service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:57.168860 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b716b-35ae-486c-ac74-57fd87c09001 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:57.174690 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7247a4b-3f50-4c31-9c21-010529ddd83f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:57.179592 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7247a4b-3f50-4c31-9c21-010529ddd83f service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:57.230757 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 904/1783] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:40:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:40:57.235710 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7247a4b-3f50-4c31-9c21-010529ddd83f service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:57.237130 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b7247a4b-3f50-4c31-9c21-010529ddd83f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:57.313577 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7247a4b-3f50-4c31-9c21-010529ddd83f service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:57.327494 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae2050-6687-42f6-963a-ca9f0a0097ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:57.330234 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae2050-6687-42f6-963a-ca9f0a0097ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:57.330695 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae2050-6687-42f6-963a-ca9f0a0097ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:57.331151 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae2050-6687-42f6-963a-ca9f0a0097ca admin admin] PATH_INFO: `/v3/projects/3280bb930e9e403894457959c81e8ba5` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:57.336484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ae2050-6687-42f6-963a-ca9f0a0097ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=3280bb930e9e403894457959c81e8ba5)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:57.340537 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ae2050-6687-42f6-963a-ca9f0a0097ca admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:57.400371 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9b716b-35ae-486c-ac74-57fd87c09001 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:57.401986 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9b716b-35ae-486c-ac74-57fd87c09001 service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:57.429714 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 880/1784] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:40:56 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:40:57.488807 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:57.545868 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a2ae2050-6687-42f6-963a-ca9f0a0097ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3280bb930e9e403894457959c81e8ba5', 'request_id': 'req-a2ae2050-6687-42f6-963a-ca9f0a0097ca'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:57.564045 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a2ae2050-6687-42f6-963a-ca9f0a0097ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3280bb930e9e403894457959c81e8ba5', 'request_id': 'req-a2ae2050-6687-42f6-963a-ca9f0a0097ca'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:57.598527 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a2ae2050-6687-42f6-963a-ca9f0a0097ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3280bb930e9e403894457959c81e8ba5', 'request_id': 'req-a2ae2050-6687-42f6-963a-ca9f0a0097ca'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:57.605096 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9b716b-35ae-486c-ac74-57fd87c09001 service cinder] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:57.616264 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9b716b-35ae-486c-ac74-57fd87c09001 service cinder] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:57.632291 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a2ae2050-6687-42f6-963a-ca9f0a0097ca admin admin] The token cache is being invalidate because project 3280bb930e9e403894457959c81e8ba5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:40:57.632619 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a2ae2050-6687-42f6-963a-ca9f0a0097ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:57.633528 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a2ae2050-6687-42f6-963a-ca9f0a0097ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:57.637843 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 905/1785] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:40:57 2022] DELETE /identity/v3/projects/3280bb930e9e403894457959c81e8ba5 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:40:57.642150 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80731c80-1699-431f-914b-7776bf36b53a None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:57.642515 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80731c80-1699-431f-914b-7776bf36b53a None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:57.642780 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80731c80-1699-431f-914b-7776bf36b53a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:57.675775 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-80731c80-1699-431f-914b-7776bf36b53a None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:57.681822 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-80731c80-1699-431f-914b-7776bf36b53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:57.746398 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efea49ad-36e6-4381-b5d3-cc205dfc4f45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:57.750111 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea49ad-36e6-4381-b5d3-cc205dfc4f45 service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:57.750657 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea49ad-36e6-4381-b5d3-cc205dfc4f45 service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:57.751110 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea49ad-36e6-4381-b5d3-cc205dfc4f45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:57.757444 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-efea49ad-36e6-4381-b5d3-cc205dfc4f45 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:57.793312 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 906/1786] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:57.799909 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02bea0e-bc3d-48c9-8fe4-7fcd48ca5d2f None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:57.800270 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02bea0e-bc3d-48c9-8fe4-7fcd48ca5d2f None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:57.800659 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02bea0e-bc3d-48c9-8fe4-7fcd48ca5d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:57.857373 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-e02bea0e-bc3d-48c9-8fe4-7fcd48ca5d2f None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:57.862662 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-e02bea0e-bc3d-48c9-8fe4-7fcd48ca5d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:57.888837 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 907/1787] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:57.894085 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02bea0e-bc3d-48c9-8fe4-7fcd48ca5d2f None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:57.899831 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efea49ad-36e6-4381-b5d3-cc205dfc4f45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:57.903689 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efea49ad-36e6-4381-b5d3-cc205dfc4f45 service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:57.913113 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02bea0e-bc3d-48c9-8fe4-7fcd48ca5d2f None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:57.923239 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f31317-8996-45a2-942f-0c7c905e8fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:57.926147 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f31317-8996-45a2-942f-0c7c905e8fae admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:57.926435 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f31317-8996-45a2-942f-0c7c905e8fae admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:57.926713 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f31317-8996-45a2-942f-0c7c905e8fae admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:57.928119 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f31317-8996-45a2-942f-0c7c905e8fae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:40:57.928432 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f31317-8996-45a2-942f-0c7c905e8fae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:57.932677 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f31317-8996-45a2-942f-0c7c905e8fae admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:57.940887 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 881/1788] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:40:57 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:40:57.983917 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 908/1789] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:40:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:40:57.987906 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efea49ad-36e6-4381-b5d3-cc205dfc4f45 service cinder] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:58.011066 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efea49ad-36e6-4381-b5d3-cc205dfc4f45 service cinder] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:58.025206 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90943ee-01d4-4557-892e-9ab6eb801f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:58.026039 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90943ee-01d4-4557-892e-9ab6eb801f0d admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:58.026408 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90943ee-01d4-4557-892e-9ab6eb801f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:58.026780 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90943ee-01d4-4557-892e-9ab6eb801f0d admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:58.027572 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90943ee-01d4-4557-892e-9ab6eb801f0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:58.029658 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90943ee-01d4-4557-892e-9ab6eb801f0d admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:58.059100 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:58.117280 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 882/1790] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:40:57 2022] POST /identity/v3/projects => generated 389 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:40:58.121126 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f31317-8996-45a2-942f-0c7c905e8fae admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:58.139709 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f31317-8996-45a2-942f-0c7c905e8fae admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:58.261454 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa49f806-de2c-4566-a373-c7a53a327a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:58.262601 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa49f806-de2c-4566-a373-c7a53a327a6f admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:58.263363 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa49f806-de2c-4566-a373-c7a53a327a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:58.263853 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa49f806-de2c-4566-a373-c7a53a327a6f admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:58.264868 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa49f806-de2c-4566-a373-c7a53a327a6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:58.268328 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa49f806-de2c-4566-a373-c7a53a327a6f admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:58.305219 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 909/1791] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:40:58 2022] POST /identity/v3/users => generated 295 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:40:58.309276 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90943ee-01d4-4557-892e-9ab6eb801f0d admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:58.324953 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90943ee-01d4-4557-892e-9ab6eb801f0d admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:58.332130 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2d1df2-2fe8-4a3a-9086-6706117c482f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:58.333727 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d1df2-2fe8-4a3a-9086-6706117c482f admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:58.334526 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d1df2-2fe8-4a3a-9086-6706117c482f admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:58.343751 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d1df2-2fe8-4a3a-9086-6706117c482f admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:58.344613 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2d1df2-2fe8-4a3a-9086-6706117c482f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:58.346678 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2d1df2-2fe8-4a3a-9086-6706117c482f admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:58.378674 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 883/1792] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:40:58 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:40:58.383246 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa49f806-de2c-4566-a373-c7a53a327a6f admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:58.399457 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa49f806-de2c-4566-a373-c7a53a327a6f admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:58.411879 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ba0440-5b73-4887-b416-a8117ec6c486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:58.414418 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba0440-5b73-4887-b416-a8117ec6c486 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:58.415034 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba0440-5b73-4887-b416-a8117ec6c486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:58.415438 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba0440-5b73-4887-b416-a8117ec6c486 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:58.416604 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ba0440-5b73-4887-b416-a8117ec6c486 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:58.419153 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ba0440-5b73-4887-b416-a8117ec6c486 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:58.440929 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 910/1793] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:40:58 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:40:58.446302 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2d1df2-2fe8-4a3a-9086-6706117c482f admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:58.467645 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2d1df2-2fe8-4a3a-9086-6706117c482f admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:58.480956 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac5fd49-baf6-43f9-ae2c-1fd4e748e884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:58.481893 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5fd49-baf6-43f9-ae2c-1fd4e748e884 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:58.482276 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5fd49-baf6-43f9-ae2c-1fd4e748e884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:58.482560 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5fd49-baf6-43f9-ae2c-1fd4e748e884 admin admin] PATH_INFO: `/v3/projects/cb82ad77bdfd429e85952a1eadc16545/users/84d53400f24f41cf88a345fd211230cf/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:58.544126 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac5fd49-baf6-43f9-ae2c-1fd4e748e884 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb82ad77bdfd429e85952a1eadc16545, user_id=84d53400f24f41cf88a345fd211230cf, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:58.547817 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac5fd49-baf6-43f9-ae2c-1fd4e748e884 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:58.583945 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 884/1794] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:40:58 2022] PUT /identity/v3/projects/cb82ad77bdfd429e85952a1eadc16545/users/84d53400f24f41cf88a345fd211230cf/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:40:58.596327 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54979e07-fff0-4e7a-b4c8-3b34f7a71539 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:58.597031 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54979e07-fff0-4e7a-b4c8-3b34f7a71539 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:58.598074 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54979e07-fff0-4e7a-b4c8-3b34f7a71539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:58.690751 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-54979e07-fff0-4e7a-b4c8-3b34f7a71539 None None] MFA Rules not processed for user `84d53400f24f41cf88a345fd211230cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:58.698305 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-54979e07-fff0-4e7a-b4c8-3b34f7a71539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:58.717996 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:58.907015 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 911/1795] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:58.908763 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac5fd49-baf6-43f9-ae2c-1fd4e748e884 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:58.995875 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54979e07-fff0-4e7a-b4c8-3b34f7a71539 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:58.998055 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-54979e07-fff0-4e7a-b4c8-3b34f7a71539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:59.141153 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac5fd49-baf6-43f9-ae2c-1fd4e748e884 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:59.162406 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2022e21c-2a85-4647-8f26-95392dc64047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:59.163092 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2022e21c-2a85-4647-8f26-95392dc64047 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:59.168894 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2022e21c-2a85-4647-8f26-95392dc64047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:59.169199 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2022e21c-2a85-4647-8f26-95392dc64047 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:59.171045 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2022e21c-2a85-4647-8f26-95392dc64047 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:59.180885 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2022e21c-2a85-4647-8f26-95392dc64047 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:59.223064 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54979e07-fff0-4e7a-b4c8-3b34f7a71539 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:59.236600 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37428c72-22b3-4e5c-a958-22f2c0c430ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:59.237833 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37428c72-22b3-4e5c-a958-22f2c0c430ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:59.238200 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37428c72-22b3-4e5c-a958-22f2c0c430ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:59.238607 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37428c72-22b3-4e5c-a958-22f2c0c430ba admin admin] PATH_INFO: `/v3/projects/546b8d65c8784cf4aa352b62557a517f` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:59.241582 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37428c72-22b3-4e5c-a958-22f2c0c430ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=546b8d65c8784cf4aa352b62557a517f)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:59.243751 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37428c72-22b3-4e5c-a958-22f2c0c430ba admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:59.265959 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 885/1796] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:40:58 2022] POST /identity/v3/projects => generated 389 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:40:59.270900 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2022e21c-2a85-4647-8f26-95392dc64047 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:59.376542 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-37428c72-22b3-4e5c-a958-22f2c0c430ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '546b8d65c8784cf4aa352b62557a517f', 'request_id': 'req-37428c72-22b3-4e5c-a958-22f2c0c430ba'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:59.388290 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-37428c72-22b3-4e5c-a958-22f2c0c430ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '546b8d65c8784cf4aa352b62557a517f', 'request_id': 'req-37428c72-22b3-4e5c-a958-22f2c0c430ba'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:59.404168 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-37428c72-22b3-4e5c-a958-22f2c0c430ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '546b8d65c8784cf4aa352b62557a517f', 'request_id': 'req-37428c72-22b3-4e5c-a958-22f2c0c430ba'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:59.432080 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-37428c72-22b3-4e5c-a958-22f2c0c430ba admin admin] The token cache is being invalidate because project 546b8d65c8784cf4aa352b62557a517f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:40:59.432343 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-37428c72-22b3-4e5c-a958-22f2c0c430ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:59.433032 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-37428c72-22b3-4e5c-a958-22f2c0c430ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:40:59.438236 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 912/1797] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:40:58 2022] DELETE /identity/v3/projects/546b8d65c8784cf4aa352b62557a517f => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:40:59.440317 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51c0b9-831b-480f-a934-1f9a0d9d795e None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:59.440623 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51c0b9-831b-480f-a934-1f9a0d9d795e None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:59.440900 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51c0b9-831b-480f-a934-1f9a0d9d795e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:59.481078 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-0a51c0b9-831b-480f-a934-1f9a0d9d795e None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:59.498934 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-0a51c0b9-831b-480f-a934-1f9a0d9d795e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:59.524209 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2022e21c-2a85-4647-8f26-95392dc64047 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:59.658998 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:59.742858 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 913/1798] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:59.743084 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:59.746823 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4bbd2-8043-4791-9078-3a250a8a44ff None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:59.747014 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4bbd2-8043-4791-9078-3a250a8a44ff None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:59.747401 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4bbd2-8043-4791-9078-3a250a8a44ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:59.786650 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ecbf34-2937-4dcd-ad21-c9ffad348586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:59.787710 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ecbf34-2937-4dcd-ad21-c9ffad348586 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:59.788161 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ecbf34-2937-4dcd-ad21-c9ffad348586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:59.788523 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ecbf34-2937-4dcd-ad21-c9ffad348586 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:59.790000 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ecbf34-2937-4dcd-ad21-c9ffad348586 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:59.794652 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ecbf34-2937-4dcd-ad21-c9ffad348586 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:59.870553 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-90f4bbd2-8043-4791-9078-3a250a8a44ff None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:40:59.878808 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-90f4bbd2-8043-4791-9078-3a250a8a44ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:59.891057 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 886/1799] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:40:59 2022] POST /identity/v3/users => generated 301 bytes in 618 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:40:59.892342 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ecbf34-2937-4dcd-ad21-c9ffad348586 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:59.902769 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-a6ecbf34-2937-4dcd-ad21-c9ffad348586 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:40:59.925134 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 914/1800] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:40:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:40:59.930390 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ecbf34-2937-4dcd-ad21-c9ffad348586 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:59.932789 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f4bbd2-8043-4791-9078-3a250a8a44ff None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:40:59.936327 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c3eaa6-c969-415a-8d9c-c44fb0345706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:40:59.937876 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c3eaa6-c969-415a-8d9c-c44fb0345706 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:40:59.938253 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c3eaa6-c969-415a-8d9c-c44fb0345706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:40:59.938651 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c3eaa6-c969-415a-8d9c-c44fb0345706 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:40:59.948478 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c3eaa6-c969-415a-8d9c-c44fb0345706 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:40:59.951306 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c3eaa6-c969-415a-8d9c-c44fb0345706 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:40:59.975574 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f4bbd2-8043-4791-9078-3a250a8a44ff None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:40:59.980802 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 887/1801] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:40:59 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:40:59.982058 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c3eaa6-c969-415a-8d9c-c44fb0345706 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:00.002459 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeda6305-23da-4333-8468-4ebab3e191d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:00.003947 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeda6305-23da-4333-8468-4ebab3e191d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:00.004246 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeda6305-23da-4333-8468-4ebab3e191d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:00.007090 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeda6305-23da-4333-8468-4ebab3e191d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:00.008699 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c3eaa6-c969-415a-8d9c-c44fb0345706 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:00.011563 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeda6305-23da-4333-8468-4ebab3e191d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:41:00.011901 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeda6305-23da-4333-8468-4ebab3e191d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:00.022087 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeda6305-23da-4333-8468-4ebab3e191d9 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:00.026334 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e951b075-4216-4b2c-9cef-70b5200104ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:00.027274 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e951b075-4216-4b2c-9cef-70b5200104ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:00.027659 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e951b075-4216-4b2c-9cef-70b5200104ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:00.028032 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e951b075-4216-4b2c-9cef-70b5200104ef admin admin] PATH_INFO: `/v3/projects/9f363be1026a4f739922fe727e716dc9/users/e1813f1fd7ea47f9bc9ec1ab93d23ec1/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:00.061682 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 915/1802] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:40:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:41:00.061682 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeda6305-23da-4333-8468-4ebab3e191d9 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:00.084725 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e951b075-4216-4b2c-9cef-70b5200104ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f363be1026a4f739922fe727e716dc9, user_id=e1813f1fd7ea47f9bc9ec1ab93d23ec1, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:00.088274 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeda6305-23da-4333-8468-4ebab3e191d9 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:00.092751 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e951b075-4216-4b2c-9cef-70b5200104ef admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:00.102313 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45493c32-5e89-45be-9df9-bef709da9bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:00.103550 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45493c32-5e89-45be-9df9-bef709da9bfc admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:00.104059 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45493c32-5e89-45be-9df9-bef709da9bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:00.104455 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45493c32-5e89-45be-9df9-bef709da9bfc admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:00.105511 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45493c32-5e89-45be-9df9-bef709da9bfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:00.109348 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45493c32-5e89-45be-9df9-bef709da9bfc admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:00.124736 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 888/1803] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:40:59 2022] PUT /identity/v3/projects/9f363be1026a4f739922fe727e716dc9/users/e1813f1fd7ea47f9bc9ec1ab93d23ec1/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:00.126212 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e951b075-4216-4b2c-9cef-70b5200104ef admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:00.206317 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 916/1804] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:41:00 2022] POST /identity/v3/projects => generated 373 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:00.211826 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45493c32-5e89-45be-9df9-bef709da9bfc admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:00.404181 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e951b075-4216-4b2c-9cef-70b5200104ef admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:00.499107 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45493c32-5e89-45be-9df9-bef709da9bfc admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:00.516455 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f3fb3f-c88e-440d-9e62-3289b755e11a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:00.518066 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3fb3f-c88e-440d-9e62-3289b755e11a admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:00.519404 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3fb3f-c88e-440d-9e62-3289b755e11a admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:00.522112 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3fb3f-c88e-440d-9e62-3289b755e11a admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:00.524340 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f3fb3f-c88e-440d-9e62-3289b755e11a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:00.530968 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f3fb3f-c88e-440d-9e62-3289b755e11a admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:00.601464 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 917/1805] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:41:00 2022] POST /identity/v3/users => generated 287 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:00.609286 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f30554-3ce8-4ec4-b76d-a6e08d886c13 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:00.614186 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f30554-3ce8-4ec4-b76d-a6e08d886c13 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:00.615314 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f30554-3ce8-4ec4-b76d-a6e08d886c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:00.633362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0c7e2d-4d8d-4068-b5eb-4d2d56acb3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:00.635929 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c7e2d-4d8d-4068-b5eb-4d2d56acb3b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:00.643733 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c7e2d-4d8d-4068-b5eb-4d2d56acb3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:00.644178 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c7e2d-4d8d-4068-b5eb-4d2d56acb3b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:00.650348 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0c7e2d-4d8d-4068-b5eb-4d2d56acb3b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:00.656651 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0c7e2d-4d8d-4068-b5eb-4d2d56acb3b0 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:00.701333 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-73f30554-3ce8-4ec4-b76d-a6e08d886c13 None None] MFA Rules not processed for user `71e0fd73e5c9430c813af1de9ccd4a52`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:00.709925 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 889/1806] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:00 2022] GET /identity/v3/roles => generated 1420 bytes in 584 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:00.715820 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf69fb2-d07b-47d2-a4f5-119365fd6b9e None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:00.719112 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf69fb2-d07b-47d2-a4f5-119365fd6b9e None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:00.719552 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf69fb2-d07b-47d2-a4f5-119365fd6b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:00.721278 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-73f30554-3ce8-4ec4-b76d-a6e08d886c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:00.796745 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-cbf69fb2-d07b-47d2-a4f5-119365fd6b9e None None] MFA Rules not processed for user `85d9c4c57681447c898ebfd7990b05b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:00.800190 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-cbf69fb2-d07b-47d2-a4f5-119365fd6b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:00.991162 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:00.996503 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 918/1807] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 394 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:01.006776 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f30554-3ce8-4ec4-b76d-a6e08d886c13 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:01.076184 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f30554-3ce8-4ec4-b76d-a6e08d886c13 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:01.081929 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 890/1808] 10.222.0.13 () {58 vars in 1009 bytes} [Fri Jun 24 08:41:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:01.084557 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf69fb2-d07b-47d2-a4f5-119365fd6b9e None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:01.111122 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a84962-8995-4d89-b7de-04495de347d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:01.116334 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a84962-8995-4d89-b7de-04495de347d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:01.117048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a84962-8995-4d89-b7de-04495de347d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:01.125726 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a84962-8995-4d89-b7de-04495de347d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:01.139872 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a84962-8995-4d89-b7de-04495de347d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:01.141166 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf69fb2-d07b-47d2-a4f5-119365fd6b9e None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:01.143960 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a84962-8995-4d89-b7de-04495de347d0 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:01.164179 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92de3254-6666-4dd3-a05a-e0d522d12222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:01.166474 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de3254-6666-4dd3-a05a-e0d522d12222 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:01.167907 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de3254-6666-4dd3-a05a-e0d522d12222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:01.168213 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de3254-6666-4dd3-a05a-e0d522d12222 admin admin] PATH_INFO: `/v3/projects/9f363be1026a4f739922fe727e716dc9/users/e1813f1fd7ea47f9bc9ec1ab93d23ec1/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:01.172852 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92de3254-6666-4dd3-a05a-e0d522d12222 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f363be1026a4f739922fe727e716dc9, user_id=e1813f1fd7ea47f9bc9ec1ab93d23ec1, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:01.185533 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92de3254-6666-4dd3-a05a-e0d522d12222 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:01.208145 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 919/1809] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:00 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:01.213626 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a84962-8995-4d89-b7de-04495de347d0 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:01.228294 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 891/1810] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:01 2022] PUT /identity/v3/projects/9f363be1026a4f739922fe727e716dc9/users/e1813f1fd7ea47f9bc9ec1ab93d23ec1/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:01.238784 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422aec8-495d-47cc-9368-8a2019b834f1 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:01.240112 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422aec8-495d-47cc-9368-8a2019b834f1 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:01.240556 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422aec8-495d-47cc-9368-8a2019b834f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:01.263824 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a84962-8995-4d89-b7de-04495de347d0 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:01.309592 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-1422aec8-495d-47cc-9368-8a2019b834f1 None None] MFA Rules not processed for user `e1813f1fd7ea47f9bc9ec1ab93d23ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:01.311094 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-1422aec8-495d-47cc-9368-8a2019b834f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:01.424694 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:01.640197 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499adad0-e5d3-4be0-a794-40de1cc50881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:01.652561 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499adad0-e5d3-4be0-a794-40de1cc50881 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:01.652828 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499adad0-e5d3-4be0-a794-40de1cc50881 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:01.653114 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499adad0-e5d3-4be0-a794-40de1cc50881 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:01.667172 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499adad0-e5d3-4be0-a794-40de1cc50881 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:01.670393 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499adad0-e5d3-4be0-a794-40de1cc50881 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:01.715110 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 892/1811] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:01 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 479 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:01.716199 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1422aec8-495d-47cc-9368-8a2019b834f1 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:01.720207 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-1422aec8-495d-47cc-9368-8a2019b834f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:01.723614 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 920/1812] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:01 2022] GET /identity/v3/roles => generated 1420 bytes in 511 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:01.724920 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499adad0-e5d3-4be0-a794-40de1cc50881 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:01.760481 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499adad0-e5d3-4be0-a794-40de1cc50881 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:01.788283 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20177262-750d-46d0-ab48-4f2595d38459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:01.789171 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20177262-750d-46d0-ab48-4f2595d38459 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:01.789459 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20177262-750d-46d0-ab48-4f2595d38459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:01.789764 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20177262-750d-46d0-ab48-4f2595d38459 admin admin] PATH_INFO: `/v3/projects/83875d4e665744acb2e19e07608d79c3/users/9a304684b4a9402398c08f9746fcc3da/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:01.845572 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20177262-750d-46d0-ab48-4f2595d38459 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83875d4e665744acb2e19e07608d79c3, user_id=9a304684b4a9402398c08f9746fcc3da, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:01.855694 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20177262-750d-46d0-ab48-4f2595d38459 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:01.925961 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 921/1813] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:01 2022] PUT /identity/v3/projects/83875d4e665744acb2e19e07608d79c3/users/9a304684b4a9402398c08f9746fcc3da/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:01.927580 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20177262-750d-46d0-ab48-4f2595d38459 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:01.937226 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-20177262-750d-46d0-ab48-4f2595d38459 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:02.055088 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1422aec8-495d-47cc-9368-8a2019b834f1 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:02.173365 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20177262-750d-46d0-ab48-4f2595d38459 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:02.191303 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e230ccc4-ab95-4e38-89b8-e7991bc3949d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:02.191977 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230ccc4-ab95-4e38-89b8-e7991bc3949d service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:02.192263 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230ccc4-ab95-4e38-89b8-e7991bc3949d service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:02.192760 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230ccc4-ab95-4e38-89b8-e7991bc3949d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:02.258616 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eb706e-f503-4e5f-93f4-6ae0141c6283 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec8c2f113cf41a0939b81f38fbee813', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:02.259296 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eb706e-f503-4e5f-93f4-6ae0141c6283 service swift] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:02.259581 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eb706e-f503-4e5f-93f4-6ae0141c6283 service swift] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:02.259859 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eb706e-f503-4e5f-93f4-6ae0141c6283 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:02.271202 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:02.383568 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eb706e-f503-4e5f-93f4-6ae0141c6283 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:02.387154 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eb706e-f503-4e5f-93f4-6ae0141c6283 service swift] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:02.396736 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e230ccc4-ab95-4e38-89b8-e7991bc3949d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:02.398830 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e230ccc4-ab95-4e38-89b8-e7991bc3949d service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:02.399493 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 893/1814] 10.222.0.13 () {60 vars in 1384 bytes} [Fri Jun 24 08:41:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 683 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 08:41:02.401242 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c191bc5b-8697-4b91-b836-b500c4c06bf5 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:02.401242 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c191bc5b-8697-4b91-b836-b500c4c06bf5 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:02.401242 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c191bc5b-8697-4b91-b836-b500c4c06bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:02.416923 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 922/1815] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:01 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:02.417976 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e230ccc4-ab95-4e38-89b8-e7991bc3949d service cinder] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:02.419519 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-e230ccc4-ab95-4e38-89b8-e7991bc3949d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:02.441251 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-c191bc5b-8697-4b91-b836-b500c4c06bf5 None None] MFA Rules not processed for user `9a304684b4a9402398c08f9746fcc3da`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:02.450613 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-c191bc5b-8697-4b91-b836-b500c4c06bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:02.543102 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:02.605176 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e230ccc4-ab95-4e38-89b8-e7991bc3949d service cinder] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:02.630654 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15458cdd-aa2b-46e2-8c4f-a196b31494a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:02.633152 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15458cdd-aa2b-46e2-8c4f-a196b31494a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:02.633466 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15458cdd-aa2b-46e2-8c4f-a196b31494a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:02.638358 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15458cdd-aa2b-46e2-8c4f-a196b31494a2 admin admin] PATH_INFO: `/v3/users/f26dfc92f03049c69960194920e13271` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:02.642435 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15458cdd-aa2b-46e2-8c4f-a196b31494a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f26dfc92f03049c69960194920e13271)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:02.652498 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15458cdd-aa2b-46e2-8c4f-a196b31494a2 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:02.720273 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 894/1816] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:02 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:02.724650 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c191bc5b-8697-4b91-b836-b500c4c06bf5 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:02.780881 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c191bc5b-8697-4b91-b836-b500c4c06bf5 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:02.797068 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-15458cdd-aa2b-46e2-8c4f-a196b31494a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f26dfc92f03049c69960194920e13271'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:02.797821 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7cc40b-03eb-4f0a-99cf-877d52fa7ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:02.799079 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7cc40b-03eb-4f0a-99cf-877d52fa7ba9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:02.799375 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7cc40b-03eb-4f0a-99cf-877d52fa7ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:02.799823 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7cc40b-03eb-4f0a-99cf-877d52fa7ba9 admin admin] PATH_INFO: `/v3/users/84d53400f24f41cf88a345fd211230cf` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:02.805870 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7cc40b-03eb-4f0a-99cf-877d52fa7ba9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84d53400f24f41cf88a345fd211230cf)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:02.811571 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7cc40b-03eb-4f0a-99cf-877d52fa7ba9 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:02.838595 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-15458cdd-aa2b-46e2-8c4f-a196b31494a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f26dfc92f03049c69960194920e13271'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:02.868946 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-15458cdd-aa2b-46e2-8c4f-a196b31494a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f26dfc92f03049c69960194920e13271'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:02.872212 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-15458cdd-aa2b-46e2-8c4f-a196b31494a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f26dfc92f03049c69960194920e13271'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:02.875636 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-15458cdd-aa2b-46e2-8c4f-a196b31494a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f26dfc92f03049c69960194920e13271'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:02.947459 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 923/1817] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:41:02 2022] DELETE /identity/v3/users/f26dfc92f03049c69960194920e13271 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:02.948335 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15458cdd-aa2b-46e2-8c4f-a196b31494a2 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:02.952620 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-15458cdd-aa2b-46e2-8c4f-a196b31494a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:03.035857 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-fa7cc40b-03eb-4f0a-99cf-877d52fa7ba9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84d53400f24f41cf88a345fd211230cf'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:03.092779 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-fa7cc40b-03eb-4f0a-99cf-877d52fa7ba9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84d53400f24f41cf88a345fd211230cf'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:03.094336 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-fa7cc40b-03eb-4f0a-99cf-877d52fa7ba9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84d53400f24f41cf88a345fd211230cf'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:03.125135 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-fa7cc40b-03eb-4f0a-99cf-877d52fa7ba9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84d53400f24f41cf88a345fd211230cf'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:03.127178 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-fa7cc40b-03eb-4f0a-99cf-877d52fa7ba9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84d53400f24f41cf88a345fd211230cf'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:03.166986 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 895/1818] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:41:02 2022] DELETE /identity/v3/users/84d53400f24f41cf88a345fd211230cf => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:03.172515 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bba753-f061-4a0a-8f9b-05aac9f8bd5c None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:03.172799 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bba753-f061-4a0a-8f9b-05aac9f8bd5c None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:03.182839 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bba753-f061-4a0a-8f9b-05aac9f8bd5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:03.237890 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:03.252990 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15458cdd-aa2b-46e2-8c4f-a196b31494a2 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:03.270026 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-42bba753-f061-4a0a-8f9b-05aac9f8bd5c None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:03.271421 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-42bba753-f061-4a0a-8f9b-05aac9f8bd5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:03.468410 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83270094-6c1c-44f2-9e2b-69eac42e56cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:03.470442 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83270094-6c1c-44f2-9e2b-69eac42e56cc admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:03.471067 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83270094-6c1c-44f2-9e2b-69eac42e56cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:03.472238 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83270094-6c1c-44f2-9e2b-69eac42e56cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:03.475232 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83270094-6c1c-44f2-9e2b-69eac42e56cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:03.479526 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83270094-6c1c-44f2-9e2b-69eac42e56cc admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:03.492519 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 896/1819] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:03.496298 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e75de-4193-41f0-8d1b-b7fba64b9897 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:03.496508 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e75de-4193-41f0-8d1b-b7fba64b9897 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:03.496794 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e75de-4193-41f0-8d1b-b7fba64b9897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:03.522873 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:03.577227 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-e48e75de-4193-41f0-8d1b-b7fba64b9897 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:03.587258 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 924/1820] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:41:02 2022] POST /identity/v3/projects => generated 371 bytes in 631 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:03.587797 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-e48e75de-4193-41f0-8d1b-b7fba64b9897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:03.587920 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83270094-6c1c-44f2-9e2b-69eac42e56cc admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:03.590378 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-83270094-6c1c-44f2-9e2b-69eac42e56cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:03.821456 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 897/1821] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:03.825585 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48e75de-4193-41f0-8d1b-b7fba64b9897 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:03.827338 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-e48e75de-4193-41f0-8d1b-b7fba64b9897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:03.852099 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83270094-6c1c-44f2-9e2b-69eac42e56cc admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:03.876879 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e0b25-34d7-4ad0-b5db-4c04e93290f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:03.877803 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e0b25-34d7-4ad0-b5db-4c04e93290f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:03.878096 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e0b25-34d7-4ad0-b5db-4c04e93290f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:03.878371 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e0b25-34d7-4ad0-b5db-4c04e93290f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:03.879079 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4e0b25-34d7-4ad0-b5db-4c04e93290f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:03.881057 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4e0b25-34d7-4ad0-b5db-4c04e93290f6 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:03.947890 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 925/1822] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:41:03 2022] POST /identity/v3/users => generated 292 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:03.949182 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e0b25-34d7-4ad0-b5db-4c04e93290f6 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:03.971731 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e0b25-34d7-4ad0-b5db-4c04e93290f6 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:03.980886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e34fa-f68b-4bc6-b0ea-4d422e9dddfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:03.981885 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48e75de-4193-41f0-8d1b-b7fba64b9897 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:03.982820 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e34fa-f68b-4bc6-b0ea-4d422e9dddfa admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:03.982820 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e34fa-f68b-4bc6-b0ea-4d422e9dddfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:03.983136 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e34fa-f68b-4bc6-b0ea-4d422e9dddfa admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:03.986622 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5e34fa-f68b-4bc6-b0ea-4d422e9dddfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:03.990175 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5e34fa-f68b-4bc6-b0ea-4d422e9dddfa admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:04.000235 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02667d4-bd14-40f7-a467-26becfc8afa6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:04.003322 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02667d4-bd14-40f7-a467-26becfc8afa6 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:04.003617 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02667d4-bd14-40f7-a467-26becfc8afa6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:04.003899 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02667d4-bd14-40f7-a467-26becfc8afa6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:04.034664 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 926/1823] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:03 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:04.040136 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e34fa-f68b-4bc6-b0ea-4d422e9dddfa admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:04.062223 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02667d4-bd14-40f7-a467-26becfc8afa6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:04.063690 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02667d4-bd14-40f7-a467-26becfc8afa6 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:04.074586 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e34fa-f68b-4bc6-b0ea-4d422e9dddfa admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:04.107518 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65677b2c-6750-491c-ab65-6424f94d65c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:04.108673 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65677b2c-6750-491c-ab65-6424f94d65c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:04.108932 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65677b2c-6750-491c-ab65-6424f94d65c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:04.109209 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65677b2c-6750-491c-ab65-6424f94d65c8 admin admin] PATH_INFO: `/v3/projects/09cd5e4f1b0746f692106b7fc6683b51/users/14ae5910834940c3b69d2bdd64dad683/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:04.136997 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 898/1824] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:04.157569 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65677b2c-6750-491c-ab65-6424f94d65c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09cd5e4f1b0746f692106b7fc6683b51, user_id=14ae5910834940c3b69d2bdd64dad683, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:04.170251 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65677b2c-6750-491c-ab65-6424f94d65c8 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:04.234213 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 927/1825] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:04 2022] PUT /identity/v3/projects/09cd5e4f1b0746f692106b7fc6683b51/users/14ae5910834940c3b69d2bdd64dad683/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:04.238365 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02667d4-bd14-40f7-a467-26becfc8afa6 service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:04.255067 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02667d4-bd14-40f7-a467-26becfc8afa6 service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:04.409230 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:04.440671 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddf798d-1eed-4ec7-bf35-2767e3efc56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:04.441476 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf798d-1eed-4ec7-bf35-2767e3efc56e admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:04.442002 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf798d-1eed-4ec7-bf35-2767e3efc56e admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:04.442911 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf798d-1eed-4ec7-bf35-2767e3efc56e admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:04.446767 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddf798d-1eed-4ec7-bf35-2767e3efc56e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:04.453479 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddf798d-1eed-4ec7-bf35-2767e3efc56e admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:04.504109 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65677b2c-6750-491c-ab65-6424f94d65c8 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:04.512649 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 899/1826] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:04 2022] GET /identity/v3/roles => generated 1420 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:04.531003 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65677b2c-6750-491c-ab65-6424f94d65c8 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:04.553522 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9493edd-3e8c-4f1a-9dca-da15c9f77673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:04.555822 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9493edd-3e8c-4f1a-9dca-da15c9f77673 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:04.558312 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9493edd-3e8c-4f1a-9dca-da15c9f77673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:04.558740 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9493edd-3e8c-4f1a-9dca-da15c9f77673 admin admin] PATH_INFO: `/v3/projects/09cd5e4f1b0746f692106b7fc6683b51/users/14ae5910834940c3b69d2bdd64dad683/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:04.562041 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9493edd-3e8c-4f1a-9dca-da15c9f77673 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09cd5e4f1b0746f692106b7fc6683b51, user_id=14ae5910834940c3b69d2bdd64dad683, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:04.575445 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9493edd-3e8c-4f1a-9dca-da15c9f77673 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:04.624614 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 928/1827] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:04 2022] PUT /identity/v3/projects/09cd5e4f1b0746f692106b7fc6683b51/users/14ae5910834940c3b69d2bdd64dad683/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:41:04.627116 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d106ccd3-f1a0-48f4-a041-20329df40a83 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:04.627523 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d106ccd3-f1a0-48f4-a041-20329df40a83 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:04.627933 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d106ccd3-f1a0-48f4-a041-20329df40a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:04.710305 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-d106ccd3-f1a0-48f4-a041-20329df40a83 None None] MFA Rules not processed for user `14ae5910834940c3b69d2bdd64dad683`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:04.711813 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d106ccd3-f1a0-48f4-a041-20329df40a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:04.778038 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6add29ec-484f-4f5a-9955-58387bffc46a None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:04.778758 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6add29ec-484f-4f5a-9955-58387bffc46a None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:04.779190 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6add29ec-484f-4f5a-9955-58387bffc46a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:04.869345 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-6add29ec-484f-4f5a-9955-58387bffc46a None None] MFA Rules not processed for user `c5f7ba6c7faf43e7acbb43eee58d8c8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:04.870840 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-6add29ec-484f-4f5a-9955-58387bffc46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:05.043754 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 900/1828] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:04 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 418 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:05.045040 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d106ccd3-f1a0-48f4-a041-20329df40a83 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:05.055809 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 929/1829] 10.222.0.49 () {58 vars in 1015 bytes} [Fri Jun 24 08:41:04 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:05.057976 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d106ccd3-f1a0-48f4-a041-20329df40a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:05.059792 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6add29ec-484f-4f5a-9955-58387bffc46a None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:05.061383 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-6add29ec-484f-4f5a-9955-58387bffc46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:05.075260 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d106ccd3-f1a0-48f4-a041-20329df40a83 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:05.081263 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065497dc-a87f-4526-8724-611fe22ab69d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:05.083755 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065497dc-a87f-4526-8724-611fe22ab69d service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:05.084942 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065497dc-a87f-4526-8724-611fe22ab69d service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:05.085520 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065497dc-a87f-4526-8724-611fe22ab69d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:05.151289 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:05.160165 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6add29ec-484f-4f5a-9955-58387bffc46a None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:05.173206 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4b6ac5-ca6f-45ed-9b19-26a149edb3bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:05.174678 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b6ac5-ca6f-45ed-9b19-26a149edb3bd service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:05.174927 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b6ac5-ca6f-45ed-9b19-26a149edb3bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:05.175207 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b6ac5-ca6f-45ed-9b19-26a149edb3bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:05.202995 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4b6ac5-ca6f-45ed-9b19-26a149edb3bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:05.205563 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4b6ac5-ca6f-45ed-9b19-26a149edb3bd service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:05.220940 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065497dc-a87f-4526-8724-611fe22ab69d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:05.225828 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065497dc-a87f-4526-8724-611fe22ab69d service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:05.259444 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 901/1831] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:05.259670 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 930/1831] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:05 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:41:05.260401 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065497dc-a87f-4526-8724-611fe22ab69d service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:05.273329 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065497dc-a87f-4526-8724-611fe22ab69d service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:05.277922 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3102ffb-2165-48eb-8fe8-cb5cc8765cbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:05.278684 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3102ffb-2165-48eb-8fe8-cb5cc8765cbd service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:05.279195 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3102ffb-2165-48eb-8fe8-cb5cc8765cbd service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:05.279508 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3102ffb-2165-48eb-8fe8-cb5cc8765cbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:05.290982 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3102ffb-2165-48eb-8fe8-cb5cc8765cbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:05.293152 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3102ffb-2165-48eb-8fe8-cb5cc8765cbd service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:05.309485 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 902/1832] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:05.425044 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4b6ac5-ca6f-45ed-9b19-26a149edb3bd service cinder] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:05.440358 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4b6ac5-ca6f-45ed-9b19-26a149edb3bd service cinder] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:05.451116 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb747a2b-99aa-40da-9a91-1597dfeaf200 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:05.453108 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb747a2b-99aa-40da-9a91-1597dfeaf200 service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:05.454813 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb747a2b-99aa-40da-9a91-1597dfeaf200 service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:05.455114 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb747a2b-99aa-40da-9a91-1597dfeaf200 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:05.461081 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-fb747a2b-99aa-40da-9a91-1597dfeaf200 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:05.515920 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3102ffb-2165-48eb-8fe8-cb5cc8765cbd service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:05.517474 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-b3102ffb-2165-48eb-8fe8-cb5cc8765cbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:05.553049 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3102ffb-2165-48eb-8fe8-cb5cc8765cbd service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:05.578939 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00866b38-f497-4747-9ba5-dd19da2f250f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:05.584820 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00866b38-f497-4747-9ba5-dd19da2f250f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:05.585321 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00866b38-f497-4747-9ba5-dd19da2f250f admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:05.588485 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00866b38-f497-4747-9ba5-dd19da2f250f admin admin] PATH_INFO: `/v3/projects/4529288ee9774d0b83e8a2b74567f502` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:05.594061 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00866b38-f497-4747-9ba5-dd19da2f250f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4529288ee9774d0b83e8a2b74567f502)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:05.601171 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00866b38-f497-4747-9ba5-dd19da2f250f admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:05.603428 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb747a2b-99aa-40da-9a91-1597dfeaf200 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:05.605039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb747a2b-99aa-40da-9a91-1597dfeaf200 service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:05.632744 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 931/1833] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:05 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:41:05.838911 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:05.847062 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-00866b38-f497-4747-9ba5-dd19da2f250f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4529288ee9774d0b83e8a2b74567f502', 'request_id': 'req-00866b38-f497-4747-9ba5-dd19da2f250f'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:05.865013 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-00866b38-f497-4747-9ba5-dd19da2f250f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4529288ee9774d0b83e8a2b74567f502', 'request_id': 'req-00866b38-f497-4747-9ba5-dd19da2f250f'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:05.888344 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-00866b38-f497-4747-9ba5-dd19da2f250f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4529288ee9774d0b83e8a2b74567f502', 'request_id': 'req-00866b38-f497-4747-9ba5-dd19da2f250f'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:05.921041 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-00866b38-f497-4747-9ba5-dd19da2f250f admin admin] The token cache is being invalidate because project 4529288ee9774d0b83e8a2b74567f502 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:41:05.921877 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-00866b38-f497-4747-9ba5-dd19da2f250f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:05.924435 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-00866b38-f497-4747-9ba5-dd19da2f250f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:05.930519 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 903/1834] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:41:05 2022] DELETE /identity/v3/projects/4529288ee9774d0b83e8a2b74567f502 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:05.935721 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb747a2b-99aa-40da-9a91-1597dfeaf200 service cinder] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:05.963403 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb747a2b-99aa-40da-9a91-1597dfeaf200 service cinder] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:06.125135 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8d0c21-6d52-4c9e-8a0a-c00951c99968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:06.126586 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d0c21-6d52-4c9e-8a0a-c00951c99968 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:06.127413 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d0c21-6d52-4c9e-8a0a-c00951c99968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:06.129025 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d0c21-6d52-4c9e-8a0a-c00951c99968 admin admin] PATH_INFO: `/v3/users/bad78bfdc12045a482c864908001b4a2` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:06.131391 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8d0c21-6d52-4c9e-8a0a-c00951c99968 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bad78bfdc12045a482c864908001b4a2)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:06.133624 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8d0c21-6d52-4c9e-8a0a-c00951c99968 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:06.282831 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7f8d0c21-6d52-4c9e-8a0a-c00951c99968 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bad78bfdc12045a482c864908001b4a2'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:06.318830 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7f8d0c21-6d52-4c9e-8a0a-c00951c99968 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bad78bfdc12045a482c864908001b4a2'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:06.338923 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7f8d0c21-6d52-4c9e-8a0a-c00951c99968 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bad78bfdc12045a482c864908001b4a2'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:06.339575 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7f8d0c21-6d52-4c9e-8a0a-c00951c99968 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bad78bfdc12045a482c864908001b4a2'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:06.340440 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7f8d0c21-6d52-4c9e-8a0a-c00951c99968 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bad78bfdc12045a482c864908001b4a2'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:06.382974 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 932/1835] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:41:05 2022] DELETE /identity/v3/users/bad78bfdc12045a482c864908001b4a2 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:08.057726 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00866b38-f497-4747-9ba5-dd19da2f250f admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:08.059207 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-00866b38-f497-4747-9ba5-dd19da2f250f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:08.092916 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00866b38-f497-4747-9ba5-dd19da2f250f admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:08.394064 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07494edc-39b4-439c-9d80-13771a2be791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:08.399150 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07494edc-39b4-439c-9d80-13771a2be791 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:08.399979 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07494edc-39b4-439c-9d80-13771a2be791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:08.400605 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07494edc-39b4-439c-9d80-13771a2be791 admin admin] PATH_INFO: `/v3/projects/cb82ad77bdfd429e85952a1eadc16545` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:08.438641 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07494edc-39b4-439c-9d80-13771a2be791 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb82ad77bdfd429e85952a1eadc16545)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:08.440941 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07494edc-39b4-439c-9d80-13771a2be791 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:08.526518 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dbaac8-674e-4d33-a228-e2111ce79c34 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:08.527421 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dbaac8-674e-4d33-a228-e2111ce79c34 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:08.527868 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dbaac8-674e-4d33-a228-e2111ce79c34 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:08.530378 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 933/1836] 10.222.0.13 () {54 vars in 921 bytes} [Fri Jun 24 08:41:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:41:08.558272 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2bd84a-aea1-4d9c-8454-b48012f50d64 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:08.558968 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2bd84a-aea1-4d9c-8454-b48012f50d64 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:08.559391 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2bd84a-aea1-4d9c-8454-b48012f50d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:08.596368 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:08.606559 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-07494edc-39b4-439c-9d80-13771a2be791 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb82ad77bdfd429e85952a1eadc16545', 'request_id': 'req-07494edc-39b4-439c-9d80-13771a2be791'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:08.617491 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-7a2bd84a-aea1-4d9c-8454-b48012f50d64 None None] MFA Rules not processed for user `74bb1bc6821b4500b26f160ccae5b7d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:08.621735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-7a2bd84a-aea1-4d9c-8454-b48012f50d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:08.633177 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-07494edc-39b4-439c-9d80-13771a2be791 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb82ad77bdfd429e85952a1eadc16545', 'request_id': 'req-07494edc-39b4-439c-9d80-13771a2be791'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:08.663514 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-07494edc-39b4-439c-9d80-13771a2be791 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb82ad77bdfd429e85952a1eadc16545', 'request_id': 'req-07494edc-39b4-439c-9d80-13771a2be791'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:08.693493 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-07494edc-39b4-439c-9d80-13771a2be791 admin admin] The token cache is being invalidate because project cb82ad77bdfd429e85952a1eadc16545 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:41:08.696544 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-07494edc-39b4-439c-9d80-13771a2be791 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:08.698846 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-07494edc-39b4-439c-9d80-13771a2be791 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:08.710177 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 904/1837] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:41:08 2022] DELETE /identity/v3/projects/cb82ad77bdfd429e85952a1eadc16545 => generated 0 bytes in 649 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:08.711510 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07494edc-39b4-439c-9d80-13771a2be791 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:08.718371 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-07494edc-39b4-439c-9d80-13771a2be791 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:08.850684 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 934/1838] 10.222.0.13 () {58 vars in 1018 bytes} [Fri Jun 24 08:41:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:09.005591 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07494edc-39b4-439c-9d80-13771a2be791 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:09.026615 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa6034b-f32b-44d7-adfb-971f440f7a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:09.026615 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa6034b-f32b-44d7-adfb-971f440f7a9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:09.026926 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa6034b-f32b-44d7-adfb-971f440f7a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:09.027016 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa6034b-f32b-44d7-adfb-971f440f7a9f admin admin] PATH_INFO: `/v3/users/e1813f1fd7ea47f9bc9ec1ab93d23ec1` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:09.028263 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa6034b-f32b-44d7-adfb-971f440f7a9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1813f1fd7ea47f9bc9ec1ab93d23ec1)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:09.031269 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa6034b-f32b-44d7-adfb-971f440f7a9f admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:09.039295 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:09.162670 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-0fa6034b-f32b-44d7-adfb-971f440f7a9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1813f1fd7ea47f9bc9ec1ab93d23ec1'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:09.199743 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-0fa6034b-f32b-44d7-adfb-971f440f7a9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1813f1fd7ea47f9bc9ec1ab93d23ec1'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:09.200562 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-0fa6034b-f32b-44d7-adfb-971f440f7a9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1813f1fd7ea47f9bc9ec1ab93d23ec1'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:09.210994 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-0fa6034b-f32b-44d7-adfb-971f440f7a9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1813f1fd7ea47f9bc9ec1ab93d23ec1'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:09.212119 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-0fa6034b-f32b-44d7-adfb-971f440f7a9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1813f1fd7ea47f9bc9ec1ab93d23ec1'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:09.254247 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 905/1839] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:41:08 2022] DELETE /identity/v3/users/e1813f1fd7ea47f9bc9ec1ab93d23ec1 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:09.326644 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2bd84a-aea1-4d9c-8454-b48012f50d64 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:09.328300 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-7a2bd84a-aea1-4d9c-8454-b48012f50d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:09.387485 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2bd84a-aea1-4d9c-8454-b48012f50d64 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:09.550862 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e651c8-11bb-4f3b-826b-c6ab4a2fe0fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74bb1bc6821b4500b26f160ccae5b7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:09.552976 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e651c8-11bb-4f3b-826b-c6ab4a2fe0fc service nova] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:09.553833 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e651c8-11bb-4f3b-826b-c6ab4a2fe0fc service nova] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:09.554783 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e651c8-11bb-4f3b-826b-c6ab4a2fe0fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:09.557130 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-85e651c8-11bb-4f3b-826b-c6ab4a2fe0fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:09.574078 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e651c8-11bb-4f3b-826b-c6ab4a2fe0fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:09.578239 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e651c8-11bb-4f3b-826b-c6ab4a2fe0fc service nova] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:09.619708 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 935/1840] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:10.465269 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa6034b-f32b-44d7-adfb-971f440f7a9f admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:10.467157 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-0fa6034b-f32b-44d7-adfb-971f440f7a9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:10.639891 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa6034b-f32b-44d7-adfb-971f440f7a9f admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:10.648128 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc3f7ea-c8cc-431a-844e-e6871e045133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:10.649027 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3f7ea-c8cc-431a-844e-e6871e045133 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:10.649317 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3f7ea-c8cc-431a-844e-e6871e045133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:10.649601 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3f7ea-c8cc-431a-844e-e6871e045133 admin admin] PATH_INFO: `/v3/projects/70497f6317a24cd18bb12d6e9779064c` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:10.656111 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc3f7ea-c8cc-431a-844e-e6871e045133 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70497f6317a24cd18bb12d6e9779064c)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:10.658188 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc3f7ea-c8cc-431a-844e-e6871e045133 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:10.749094 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7bc3f7ea-c8cc-431a-844e-e6871e045133 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70497f6317a24cd18bb12d6e9779064c', 'request_id': 'req-7bc3f7ea-c8cc-431a-844e-e6871e045133'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:10.757392 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7bc3f7ea-c8cc-431a-844e-e6871e045133 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70497f6317a24cd18bb12d6e9779064c', 'request_id': 'req-7bc3f7ea-c8cc-431a-844e-e6871e045133'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:10.765768 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7bc3f7ea-c8cc-431a-844e-e6871e045133 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70497f6317a24cd18bb12d6e9779064c', 'request_id': 'req-7bc3f7ea-c8cc-431a-844e-e6871e045133'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:10.787852 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7bc3f7ea-c8cc-431a-844e-e6871e045133 admin admin] The token cache is being invalidate because project 70497f6317a24cd18bb12d6e9779064c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:41:10.788382 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7bc3f7ea-c8cc-431a-844e-e6871e045133 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:10.789514 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7bc3f7ea-c8cc-431a-844e-e6871e045133 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:10.791687 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 906/1841] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:41:10 2022] DELETE /identity/v3/projects/70497f6317a24cd18bb12d6e9779064c => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:10.796994 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0367da07-7e47-4ba1-8644-a27909eb57f2 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:10.797438 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0367da07-7e47-4ba1-8644-a27909eb57f2 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:10.797569 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0367da07-7e47-4ba1-8644-a27909eb57f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:10.836090 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-0367da07-7e47-4ba1-8644-a27909eb57f2 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:10.837654 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-0367da07-7e47-4ba1-8644-a27909eb57f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:10.941008 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 936/1842] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:10.949511 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f152eff-771c-4c6a-b943-9b3f2bb54682 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:10.950232 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f152eff-771c-4c6a-b943-9b3f2bb54682 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:10.950656 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f152eff-771c-4c6a-b943-9b3f2bb54682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:10.992453 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-4f152eff-771c-4c6a-b943-9b3f2bb54682 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:10.994168 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-4f152eff-771c-4c6a-b943-9b3f2bb54682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:11.007166 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 907/1843] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:11.011703 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0367da07-7e47-4ba1-8644-a27909eb57f2 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:11.033960 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f152eff-771c-4c6a-b943-9b3f2bb54682 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:11.038248 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0367da07-7e47-4ba1-8644-a27909eb57f2 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:11.039047 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-4f152eff-771c-4c6a-b943-9b3f2bb54682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:11.050053 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e2a18-866e-43bd-b83c-c4484c2a8af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:11.051190 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e2a18-866e-43bd-b83c-c4484c2a8af3 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:11.053535 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e2a18-866e-43bd-b83c-c4484c2a8af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:11.054501 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e2a18-866e-43bd-b83c-c4484c2a8af3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:11.057441 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89e2a18-866e-43bd-b83c-c4484c2a8af3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:41:11.057942 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89e2a18-866e-43bd-b83c-c4484c2a8af3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:11.060613 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89e2a18-866e-43bd-b83c-c4484c2a8af3 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:11.069681 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f152eff-771c-4c6a-b943-9b3f2bb54682 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:11.082263 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db263efa-72a6-4e4c-bfab-591ee4ee9204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:11.083502 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db263efa-72a6-4e4c-bfab-591ee4ee9204 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:11.083951 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db263efa-72a6-4e4c-bfab-591ee4ee9204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:11.084136 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db263efa-72a6-4e4c-bfab-591ee4ee9204 admin admin] PATH_INFO: `/v3/projects/9f363be1026a4f739922fe727e716dc9` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:11.087041 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db263efa-72a6-4e4c-bfab-591ee4ee9204 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f363be1026a4f739922fe727e716dc9)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:11.088960 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db263efa-72a6-4e4c-bfab-591ee4ee9204 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:11.097168 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 937/1844] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:41:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:41:11.098653 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e2a18-866e-43bd-b83c-c4484c2a8af3 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:11.104739 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:11.121134 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e2a18-866e-43bd-b83c-c4484c2a8af3 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:11.136654 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31030682-8275-4dd4-bb87-b29071dafc61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:11.137237 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31030682-8275-4dd4-bb87-b29071dafc61 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:11.137507 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31030682-8275-4dd4-bb87-b29071dafc61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:11.137803 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31030682-8275-4dd4-bb87-b29071dafc61 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:11.138493 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31030682-8275-4dd4-bb87-b29071dafc61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:11.140511 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31030682-8275-4dd4-bb87-b29071dafc61 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:11.159082 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:11.206481 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 938/1845] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:41:11 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:11.208341 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31030682-8275-4dd4-bb87-b29071dafc61 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:11.208785 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-db263efa-72a6-4e4c-bfab-591ee4ee9204 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f363be1026a4f739922fe727e716dc9', 'request_id': 'req-db263efa-72a6-4e4c-bfab-591ee4ee9204'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:11.221281 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-db263efa-72a6-4e4c-bfab-591ee4ee9204 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f363be1026a4f739922fe727e716dc9', 'request_id': 'req-db263efa-72a6-4e4c-bfab-591ee4ee9204'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:11.231352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-db263efa-72a6-4e4c-bfab-591ee4ee9204 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f363be1026a4f739922fe727e716dc9', 'request_id': 'req-db263efa-72a6-4e4c-bfab-591ee4ee9204'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:11.268526 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-db263efa-72a6-4e4c-bfab-591ee4ee9204 admin admin] The token cache is being invalidate because project 9f363be1026a4f739922fe727e716dc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:41:11.271391 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-db263efa-72a6-4e4c-bfab-591ee4ee9204 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:11.277833 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-db263efa-72a6-4e4c-bfab-591ee4ee9204 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:11.280748 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 908/1846] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:41:11 2022] DELETE /identity/v3/projects/9f363be1026a4f739922fe727e716dc9 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:11.285443 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a15d8-e632-4372-98aa-fa57f2c2b6f4 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:11.286582 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a15d8-e632-4372-98aa-fa57f2c2b6f4 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:11.286785 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a15d8-e632-4372-98aa-fa57f2c2b6f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:11.333261 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-907a15d8-e632-4372-98aa-fa57f2c2b6f4 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:11.334548 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-907a15d8-e632-4372-98aa-fa57f2c2b6f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:11.466850 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31030682-8275-4dd4-bb87-b29071dafc61 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:11.479892 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b837a667-3488-41cf-a21b-54dafe543254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:11.481608 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837a667-3488-41cf-a21b-54dafe543254 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:11.491854 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837a667-3488-41cf-a21b-54dafe543254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:11.491854 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837a667-3488-41cf-a21b-54dafe543254 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:11.491854 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b837a667-3488-41cf-a21b-54dafe543254 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:11.493563 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b837a667-3488-41cf-a21b-54dafe543254 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:11.522804 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 909/1847] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:11.530519 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb021551-a0e1-4ec5-9fea-14fe85043489 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:11.531091 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb021551-a0e1-4ec5-9fea-14fe85043489 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:11.531644 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb021551-a0e1-4ec5-9fea-14fe85043489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:11.548873 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 939/1848] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:41:11 2022] POST /identity/v3/users => generated 293 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:11.554610 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b837a667-3488-41cf-a21b-54dafe543254 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:11.558168 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b837a667-3488-41cf-a21b-54dafe543254 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:11.584270 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-bb021551-a0e1-4ec5-9fea-14fe85043489 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:11.586079 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-bb021551-a0e1-4ec5-9fea-14fe85043489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:11.587338 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b837a667-3488-41cf-a21b-54dafe543254 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:11.595139 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 910/1849] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:11.599975 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb021551-a0e1-4ec5-9fea-14fe85043489 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:11.602894 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce05dd5-750c-40f2-bfc4-ae50211c1aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:11.603747 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce05dd5-750c-40f2-bfc4-ae50211c1aeb admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:11.604128 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce05dd5-750c-40f2-bfc4-ae50211c1aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:11.604498 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce05dd5-750c-40f2-bfc4-ae50211c1aeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:11.605824 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce05dd5-750c-40f2-bfc4-ae50211c1aeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:11.607627 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce05dd5-750c-40f2-bfc4-ae50211c1aeb admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:11.612749 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb021551-a0e1-4ec5-9fea-14fe85043489 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:11.619298 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b6aa40-9769-4968-ac0b-e3194512a60f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:11.620225 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6aa40-9769-4968-ac0b-e3194512a60f admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:11.620648 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6aa40-9769-4968-ac0b-e3194512a60f admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:11.621087 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6aa40-9769-4968-ac0b-e3194512a60f admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:11.622156 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b6aa40-9769-4968-ac0b-e3194512a60f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:41:11.622869 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b6aa40-9769-4968-ac0b-e3194512a60f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:11.624678 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b6aa40-9769-4968-ac0b-e3194512a60f admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:11.634884 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 940/1850] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:11 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:41:11.636441 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce05dd5-750c-40f2-bfc4-ae50211c1aeb admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:11.658350 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce05dd5-750c-40f2-bfc4-ae50211c1aeb admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:11.659774 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 911/1851] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:41:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:41:11.664559 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b6aa40-9769-4968-ac0b-e3194512a60f admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:11.670048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632a8c3a-4d1f-4b86-af6a-c59785324844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:11.670947 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632a8c3a-4d1f-4b86-af6a-c59785324844 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:11.671324 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632a8c3a-4d1f-4b86-af6a-c59785324844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:11.671818 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632a8c3a-4d1f-4b86-af6a-c59785324844 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:11.672663 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632a8c3a-4d1f-4b86-af6a-c59785324844 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:11.674583 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632a8c3a-4d1f-4b86-af6a-c59785324844 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:11.683247 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b6aa40-9769-4968-ac0b-e3194512a60f admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:11.691434 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6659f2e-f5c5-479c-a681-82ad5fdfb55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:11.694279 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6659f2e-f5c5-479c-a681-82ad5fdfb55a admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:11.694748 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6659f2e-f5c5-479c-a681-82ad5fdfb55a admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:11.695698 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6659f2e-f5c5-479c-a681-82ad5fdfb55a admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:11.697175 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6659f2e-f5c5-479c-a681-82ad5fdfb55a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:11.698563 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 941/1852] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:11 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:11.700181 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6659f2e-f5c5-479c-a681-82ad5fdfb55a admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:11.702563 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632a8c3a-4d1f-4b86-af6a-c59785324844 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:11.716729 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632a8c3a-4d1f-4b86-af6a-c59785324844 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:11.728335 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13973ef-89e9-4ed0-84bd-e9a6c157c6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:11.729784 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13973ef-89e9-4ed0-84bd-e9a6c157c6d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:11.730132 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13973ef-89e9-4ed0-84bd-e9a6c157c6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:11.731017 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13973ef-89e9-4ed0-84bd-e9a6c157c6d5 admin admin] PATH_INFO: `/v3/projects/7479df1d1f7f433d92b43b8f6f4fab69/users/213e60e2287e4578a1e5c39f131bcbb9/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:11.740856 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 912/1853] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:41:11 2022] POST /identity/v3/projects => generated 409 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:11.746827 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6659f2e-f5c5-479c-a681-82ad5fdfb55a admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:11.754821 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13973ef-89e9-4ed0-84bd-e9a6c157c6d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7479df1d1f7f433d92b43b8f6f4fab69, user_id=213e60e2287e4578a1e5c39f131bcbb9, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:11.757872 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13973ef-89e9-4ed0-84bd-e9a6c157c6d5 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:11.777576 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 942/1854] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:11 2022] PUT /identity/v3/projects/7479df1d1f7f433d92b43b8f6f4fab69/users/213e60e2287e4578a1e5c39f131bcbb9/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:41:11.790267 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01be1f29-fe21-4fce-82bb-f4e1e39d6722 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:11.790656 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01be1f29-fe21-4fce-82bb-f4e1e39d6722 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:11.791119 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01be1f29-fe21-4fce-82bb-f4e1e39d6722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:11.823935 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:11.846954 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-01be1f29-fe21-4fce-82bb-f4e1e39d6722 None None] MFA Rules not processed for user `213e60e2287e4578a1e5c39f131bcbb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:11.853491 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-01be1f29-fe21-4fce-82bb-f4e1e39d6722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:11.856684 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6659f2e-f5c5-479c-a681-82ad5fdfb55a admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:11.934784 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8023496b-342f-4e7d-8403-69b09b4c6409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:11.935373 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8023496b-342f-4e7d-8403-69b09b4c6409 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:11.935660 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8023496b-342f-4e7d-8403-69b09b4c6409 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:11.935946 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8023496b-342f-4e7d-8403-69b09b4c6409 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:11.936645 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8023496b-342f-4e7d-8403-69b09b4c6409 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:11.938632 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8023496b-342f-4e7d-8403-69b09b4c6409 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:11.970359 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 943/1855] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:11.970359 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a3208-5f2d-4952-b2ba-690a844e1d29 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:11.970359 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a3208-5f2d-4952-b2ba-690a844e1d29 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:11.970359 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a3208-5f2d-4952-b2ba-690a844e1d29 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:11.970359 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 944/1856] 10.222.0.13 () {54 vars in 921 bytes} [Fri Jun 24 08:41:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:41:11.981287 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5b217-b96c-4e7d-b9b7-912df8cba034 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:11.982163 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5b217-b96c-4e7d-b9b7-912df8cba034 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:11.982618 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5b217-b96c-4e7d-b9b7-912df8cba034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:11.982927 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 913/1857] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:41:11 2022] POST /identity/v3/users => generated 305 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:11.983867 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8023496b-342f-4e7d-8403-69b09b4c6409 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:11.985776 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-8023496b-342f-4e7d-8403-69b09b4c6409 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:12.007472 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-5aa5b217-b96c-4e7d-b9b7-912df8cba034 None None] MFA Rules not processed for user `74bb1bc6821b4500b26f160ccae5b7d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:12.009111 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-5aa5b217-b96c-4e7d-b9b7-912df8cba034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:12.049471 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:12.073780 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8023496b-342f-4e7d-8403-69b09b4c6409 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:12.077587 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1ceab3-2c17-4453-8460-bbac29649c4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:12.078298 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ceab3-2c17-4453-8460-bbac29649c4a service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:12.078574 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ceab3-2c17-4453-8460-bbac29649c4a service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:12.078850 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ceab3-2c17-4453-8460-bbac29649c4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:12.090088 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1ceab3-2c17-4453-8460-bbac29649c4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:12.091555 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1ceab3-2c17-4453-8460-bbac29649c4a service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:12.102991 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 914/1858] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:12.103675 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1ceab3-2c17-4453-8460-bbac29649c4a service cinder] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:12.117738 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1ceab3-2c17-4453-8460-bbac29649c4a service cinder] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:12.119676 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 945/1859] 10.222.0.13 () {58 vars in 1018 bytes} [Fri Jun 24 08:41:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:12.124634 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd055c9-1626-4740-b909-4b60b7d24e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:12.125344 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd055c9-1626-4740-b909-4b60b7d24e95 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:12.125806 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd055c9-1626-4740-b909-4b60b7d24e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:12.126118 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd055c9-1626-4740-b909-4b60b7d24e95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:12.126897 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd055c9-1626-4740-b909-4b60b7d24e95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:12.128430 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd055c9-1626-4740-b909-4b60b7d24e95 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:12.132116 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa5b217-b96c-4e7d-b9b7-912df8cba034 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:12.144747 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa5b217-b96c-4e7d-b9b7-912df8cba034 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:12.152311 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 915/1860] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:12 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:12.153380 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd055c9-1626-4740-b909-4b60b7d24e95 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:12.154763 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0709628-0f81-4e27-bab7-84522a6e723e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:12.156087 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0709628-0f81-4e27-bab7-84522a6e723e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:12.156819 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0709628-0f81-4e27-bab7-84522a6e723e admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:12.157412 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0709628-0f81-4e27-bab7-84522a6e723e admin admin] PATH_INFO: `/v3/users/213e60e2287e4578a1e5c39f131bcbb9` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:12.159340 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0709628-0f81-4e27-bab7-84522a6e723e admin admin] RBAC: Authorizing `identity:delete_user(user_id=213e60e2287e4578a1e5c39f131bcbb9)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:12.161657 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0709628-0f81-4e27-bab7-84522a6e723e admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:12.163947 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd055c9-1626-4740-b909-4b60b7d24e95 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:12.168940 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f329c3-f470-41bd-bd7f-16560e383a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:12.169616 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f329c3-f470-41bd-bd7f-16560e383a64 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:12.169927 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f329c3-f470-41bd-bd7f-16560e383a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:12.170205 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f329c3-f470-41bd-bd7f-16560e383a64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:12.170937 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f329c3-f470-41bd-bd7f-16560e383a64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:12.172717 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f329c3-f470-41bd-bd7f-16560e383a64 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:12.184534 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 916/1861] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:12.188176 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f329c3-f470-41bd-bd7f-16560e383a64 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:12.201282 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f329c3-f470-41bd-bd7f-16560e383a64 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:12.206954 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82122897-f770-40a2-aaa4-c86d2bfdcc50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:12.208646 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82122897-f770-40a2-aaa4-c86d2bfdcc50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:12.209102 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82122897-f770-40a2-aaa4-c86d2bfdcc50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:12.209428 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82122897-f770-40a2-aaa4-c86d2bfdcc50 admin admin] PATH_INFO: `/v3/projects/096e03d26cba4d33a3a1f272d1bb00e0/users/a5b9c4b8ec34467a8955aff85797e9ac/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:12.227831 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c0709628-0f81-4e27-bab7-84522a6e723e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '213e60e2287e4578a1e5c39f131bcbb9'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:12.228437 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82122897-f770-40a2-aaa4-c86d2bfdcc50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=096e03d26cba4d33a3a1f272d1bb00e0, user_id=a5b9c4b8ec34467a8955aff85797e9ac, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:12.231116 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82122897-f770-40a2-aaa4-c86d2bfdcc50 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:12.246484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c0709628-0f81-4e27-bab7-84522a6e723e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '213e60e2287e4578a1e5c39f131bcbb9'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:12.259225 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 917/1862] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:12 2022] PUT /identity/v3/projects/096e03d26cba4d33a3a1f272d1bb00e0/users/a5b9c4b8ec34467a8955aff85797e9ac/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:41:12.272239 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967b1300-bb83-4edd-a178-dd0c090ce958 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:12.272239 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967b1300-bb83-4edd-a178-dd0c090ce958 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:12.273338 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967b1300-bb83-4edd-a178-dd0c090ce958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:12.276864 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c0709628-0f81-4e27-bab7-84522a6e723e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '213e60e2287e4578a1e5c39f131bcbb9'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:12.277519 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c0709628-0f81-4e27-bab7-84522a6e723e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '213e60e2287e4578a1e5c39f131bcbb9'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:12.278318 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c0709628-0f81-4e27-bab7-84522a6e723e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '213e60e2287e4578a1e5c39f131bcbb9'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:12.293116 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 946/1863] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:41:12 2022] DELETE /identity/v3/users/213e60e2287e4578a1e5c39f131bcbb9 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:41:12.294053 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0709628-0f81-4e27-bab7-84522a6e723e admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:12.295725 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-c0709628-0f81-4e27-bab7-84522a6e723e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:12.308530 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-967b1300-bb83-4edd-a178-dd0c090ce958 None None] MFA Rules not processed for user `a5b9c4b8ec34467a8955aff85797e9ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:12.310176 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-967b1300-bb83-4edd-a178-dd0c090ce958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:12.383893 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:12.427791 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0709628-0f81-4e27-bab7-84522a6e723e admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:12.435478 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5badc3-40fe-489c-89f8-683968620226 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74bb1bc6821b4500b26f160ccae5b7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:12.436409 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5badc3-40fe-489c-89f8-683968620226 service nova] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:12.436920 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5badc3-40fe-489c-89f8-683968620226 service nova] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:12.437316 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5badc3-40fe-489c-89f8-683968620226 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:12.439416 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-7d5badc3-40fe-489c-89f8-683968620226 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:12.447313 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 918/1864] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:12 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:12.449159 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ac72c-ae96-49d4-8478-0c84e0179e33 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:12.449605 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ac72c-ae96-49d4-8478-0c84e0179e33 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:12.450064 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ac72c-ae96-49d4-8478-0c84e0179e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:12.455556 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5badc3-40fe-489c-89f8-683968620226 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:12.457301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5badc3-40fe-489c-89f8-683968620226 service nova] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:12.470714 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 947/1865] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:12.471754 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5badc3-40fe-489c-89f8-683968620226 service nova] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:12.473238 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-7d5badc3-40fe-489c-89f8-683968620226 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:12.478085 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-063ac72c-ae96-49d4-8478-0c84e0179e33 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:12.480428 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-063ac72c-ae96-49d4-8478-0c84e0179e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:12.490757 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:12.578614 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 919/1866] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:12.594801 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063ac72c-ae96-49d4-8478-0c84e0179e33 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:12.596514 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-063ac72c-ae96-49d4-8478-0c84e0179e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:12.672038 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5badc3-40fe-489c-89f8-683968620226 service nova] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:12.686343 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1693e3-0ce2-4cf7-9e7a-34ad7b3ff4df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:12.689914 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1693e3-0ce2-4cf7-9e7a-34ad7b3ff4df admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:12.690610 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1693e3-0ce2-4cf7-9e7a-34ad7b3ff4df admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:12.691782 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1693e3-0ce2-4cf7-9e7a-34ad7b3ff4df admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:12.692879 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1693e3-0ce2-4cf7-9e7a-34ad7b3ff4df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:12.698737 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1693e3-0ce2-4cf7-9e7a-34ad7b3ff4df admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:12.720724 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063ac72c-ae96-49d4-8478-0c84e0179e33 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:12.738095 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce320700-3721-4f90-9b74-a69873d8e5c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:12.741639 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce320700-3721-4f90-9b74-a69873d8e5c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:12.741953 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce320700-3721-4f90-9b74-a69873d8e5c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:12.742216 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce320700-3721-4f90-9b74-a69873d8e5c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:12.758545 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce320700-3721-4f90-9b74-a69873d8e5c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:12.760936 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce320700-3721-4f90-9b74-a69873d8e5c9 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:12.770003 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 948/1867] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:41:12 2022] POST /identity/v3/projects => generated 407 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:12.775294 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1693e3-0ce2-4cf7-9e7a-34ad7b3ff4df admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:12.794895 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 920/1868] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:12.886145 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1693e3-0ce2-4cf7-9e7a-34ad7b3ff4df admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:12.894844 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ee19f7-00a8-4005-bdc4-b69b8ebfffe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:12.896409 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee19f7-00a8-4005-bdc4-b69b8ebfffe6 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:12.897134 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee19f7-00a8-4005-bdc4-b69b8ebfffe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:12.898120 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee19f7-00a8-4005-bdc4-b69b8ebfffe6 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:12.901319 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ee19f7-00a8-4005-bdc4-b69b8ebfffe6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:12.905947 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ee19f7-00a8-4005-bdc4-b69b8ebfffe6 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:12.941223 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 949/1869] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:41:12 2022] POST /identity/v3/users => generated 310 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:12.944186 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce320700-3721-4f90-9b74-a69873d8e5c9 service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:12.958347 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce320700-3721-4f90-9b74-a69873d8e5c9 service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:12.966620 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19d3093-8784-4e32-a340-89e02b3aef30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:12.967823 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19d3093-8784-4e32-a340-89e02b3aef30 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:12.968316 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19d3093-8784-4e32-a340-89e02b3aef30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:12.968790 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19d3093-8784-4e32-a340-89e02b3aef30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:12.969864 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19d3093-8784-4e32-a340-89e02b3aef30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:12.991022 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19d3093-8784-4e32-a340-89e02b3aef30 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:13.043483 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ee19f7-00a8-4005-bdc4-b69b8ebfffe6 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:13.054974 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 921/1870] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:12 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:13.069757 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ee19f7-00a8-4005-bdc4-b69b8ebfffe6 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:13.088026 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecd8c88-456b-41fe-a105-3660e7f1425f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:13.089107 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecd8c88-456b-41fe-a105-3660e7f1425f admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:13.089383 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecd8c88-456b-41fe-a105-3660e7f1425f admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:13.089642 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecd8c88-456b-41fe-a105-3660e7f1425f admin admin] PATH_INFO: `/v3/projects/ffa71ae1020e41fba96f2f5b66c0b91e/users/51c1f9409afb42ac93952aca941916b3/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:13.129096 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecd8c88-456b-41fe-a105-3660e7f1425f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffa71ae1020e41fba96f2f5b66c0b91e, user_id=51c1f9409afb42ac93952aca941916b3, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:13.132723 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecd8c88-456b-41fe-a105-3660e7f1425f admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:13.159535 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 950/1871] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:13 2022] PUT /identity/v3/projects/ffa71ae1020e41fba96f2f5b66c0b91e/users/51c1f9409afb42ac93952aca941916b3/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:41:13.161504 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19d3093-8784-4e32-a340-89e02b3aef30 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:13.182700 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19d3093-8784-4e32-a340-89e02b3aef30 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:13.265268 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b1e552-4606-41f2-9691-6b76217fb6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:13.266363 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b1e552-4606-41f2-9691-6b76217fb6e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:13.266677 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b1e552-4606-41f2-9691-6b76217fb6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:13.266948 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b1e552-4606-41f2-9691-6b76217fb6e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:13.267703 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b1e552-4606-41f2-9691-6b76217fb6e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:13.270095 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b1e552-4606-41f2-9691-6b76217fb6e7 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:13.304157 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 922/1872] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:13 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:13.305708 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecd8c88-456b-41fe-a105-3660e7f1425f admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:13.321213 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecd8c88-456b-41fe-a105-3660e7f1425f admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:13.327355 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e5fae-e981-4be5-9c3e-dc20a1693baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:13.328635 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e5fae-e981-4be5-9c3e-dc20a1693baf admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:13.328925 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e5fae-e981-4be5-9c3e-dc20a1693baf admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:13.329200 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e5fae-e981-4be5-9c3e-dc20a1693baf admin admin] PATH_INFO: `/v3/projects/ffa71ae1020e41fba96f2f5b66c0b91e/users/51c1f9409afb42ac93952aca941916b3/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:13.331378 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991e5fae-e981-4be5-9c3e-dc20a1693baf admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffa71ae1020e41fba96f2f5b66c0b91e, user_id=51c1f9409afb42ac93952aca941916b3, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:13.334363 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991e5fae-e981-4be5-9c3e-dc20a1693baf admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:13.359596 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 951/1873] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:13 2022] PUT /identity/v3/projects/ffa71ae1020e41fba96f2f5b66c0b91e/users/51c1f9409afb42ac93952aca941916b3/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:13.363379 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098d833f-c943-47b9-a5be-63739b9888a6 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:13.364387 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098d833f-c943-47b9-a5be-63739b9888a6 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:13.365112 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098d833f-c943-47b9-a5be-63739b9888a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:13.375994 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:13.436384 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-098d833f-c943-47b9-a5be-63739b9888a6 None None] MFA Rules not processed for user `51c1f9409afb42ac93952aca941916b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:13.439044 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-098d833f-c943-47b9-a5be-63739b9888a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:13.466510 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e5fae-e981-4be5-9c3e-dc20a1693baf admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:13.469885 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-991e5fae-e981-4be5-9c3e-dc20a1693baf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:13.580014 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:13.640119 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 923/1874] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:13 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:13.671884 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e5fae-e981-4be5-9c3e-dc20a1693baf admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:13.694911 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8601d27e-e9be-482f-aad0-aaed56372896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:13.696065 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8601d27e-e9be-482f-aad0-aaed56372896 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:13.696461 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8601d27e-e9be-482f-aad0-aaed56372896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:13.696958 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8601d27e-e9be-482f-aad0-aaed56372896 admin admin] PATH_INFO: `/v3/projects/7479df1d1f7f433d92b43b8f6f4fab69` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:13.698824 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8601d27e-e9be-482f-aad0-aaed56372896 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7479df1d1f7f433d92b43b8f6f4fab69)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:13.700960 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8601d27e-e9be-482f-aad0-aaed56372896 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:13.797437 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098d833f-c943-47b9-a5be-63739b9888a6 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:13.799765 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-098d833f-c943-47b9-a5be-63739b9888a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:13.891465 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-8601d27e-e9be-482f-aad0-aaed56372896 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7479df1d1f7f433d92b43b8f6f4fab69', 'request_id': 'req-8601d27e-e9be-482f-aad0-aaed56372896'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:13.914507 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-8601d27e-e9be-482f-aad0-aaed56372896 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7479df1d1f7f433d92b43b8f6f4fab69', 'request_id': 'req-8601d27e-e9be-482f-aad0-aaed56372896'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:13.936041 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-8601d27e-e9be-482f-aad0-aaed56372896 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7479df1d1f7f433d92b43b8f6f4fab69', 'request_id': 'req-8601d27e-e9be-482f-aad0-aaed56372896'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:13.984483 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098d833f-c943-47b9-a5be-63739b9888a6 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:13.988386 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-8601d27e-e9be-482f-aad0-aaed56372896 admin admin] The token cache is being invalidate because project 7479df1d1f7f433d92b43b8f6f4fab69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:41:13.989916 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-8601d27e-e9be-482f-aad0-aaed56372896 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:13.991769 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-8601d27e-e9be-482f-aad0-aaed56372896 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:13.999740 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 952/1875] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:41:13 2022] DELETE /identity/v3/projects/7479df1d1f7f433d92b43b8f6f4fab69 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:41:14.007248 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeed233-a0b4-4055-991e-ae405bb7e4c2 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:14.008973 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeed233-a0b4-4055-991e-ae405bb7e4c2 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:14.010575 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeed233-a0b4-4055-991e-ae405bb7e4c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:14.085343 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-aaeed233-a0b4-4055-991e-ae405bb7e4c2 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:14.093353 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-aaeed233-a0b4-4055-991e-ae405bb7e4c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:14.181740 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a638bb1b-9147-401b-8a25-3beb488c30ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:14.182695 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638bb1b-9147-401b-8a25-3beb488c30ac service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:14.183196 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638bb1b-9147-401b-8a25-3beb488c30ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:14.183638 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638bb1b-9147-401b-8a25-3beb488c30ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:14.188643 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-a638bb1b-9147-401b-8a25-3beb488c30ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:14.316133 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 953/1876] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:14.318905 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ca302-ac82-4568-8b1b-619323008a90 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:14.319198 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ca302-ac82-4568-8b1b-619323008a90 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:14.319544 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ca302-ac82-4568-8b1b-619323008a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:14.375977 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-c79ca302-ac82-4568-8b1b-619323008a90 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:14.381735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-c79ca302-ac82-4568-8b1b-619323008a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:14.399361 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:14.407487 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 954/1877] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:14.415203 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ca302-ac82-4568-8b1b-619323008a90 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:14.443354 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a638bb1b-9147-401b-8a25-3beb488c30ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:14.463163 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a638bb1b-9147-401b-8a25-3beb488c30ac service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:14.493319 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ca302-ac82-4568-8b1b-619323008a90 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:14.513243 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc367103-47e5-44b0-b129-93d9d3fa9f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:14.514484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc367103-47e5-44b0-b129-93d9d3fa9f61 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:14.514782 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc367103-47e5-44b0-b129-93d9d3fa9f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:14.516568 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc367103-47e5-44b0-b129-93d9d3fa9f61 admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:14.519814 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc367103-47e5-44b0-b129-93d9d3fa9f61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:41:14.520398 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc367103-47e5-44b0-b129-93d9d3fa9f61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:14.525819 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc367103-47e5-44b0-b129-93d9d3fa9f61 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:14.550789 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 924/1878] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:13 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 753 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:14.589727 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 955/1879] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:41:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:41:14.591956 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a638bb1b-9147-401b-8a25-3beb488c30ac service cinder] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:14.633862 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a638bb1b-9147-401b-8a25-3beb488c30ac service cinder] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:14.653192 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a02f241-55ee-4cfc-bd0c-5e49951a1d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:14.654417 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02f241-55ee-4cfc-bd0c-5e49951a1d12 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:14.654941 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02f241-55ee-4cfc-bd0c-5e49951a1d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:14.655498 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02f241-55ee-4cfc-bd0c-5e49951a1d12 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:14.657000 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a02f241-55ee-4cfc-bd0c-5e49951a1d12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:14.679191 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a02f241-55ee-4cfc-bd0c-5e49951a1d12 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:14.768271 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 925/1880] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:41:14 2022] POST /identity/v3/projects => generated 383 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:14.787324 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc367103-47e5-44b0-b129-93d9d3fa9f61 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:14.804645 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc367103-47e5-44b0-b129-93d9d3fa9f61 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:15.040814 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3c50a8-0285-4a32-b53c-e1c672470e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:15.043118 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c50a8-0285-4a32-b53c-e1c672470e34 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:15.044203 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c50a8-0285-4a32-b53c-e1c672470e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:15.045710 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c50a8-0285-4a32-b53c-e1c672470e34 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:15.049028 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3c50a8-0285-4a32-b53c-e1c672470e34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:15.053574 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3c50a8-0285-4a32-b53c-e1c672470e34 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:15.079054 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:15.118963 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a02f241-55ee-4cfc-bd0c-5e49951a1d12 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:15.120431 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 956/1881] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:41:14 2022] POST /identity/v3/users => generated 292 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:15.120835 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-1a02f241-55ee-4cfc-bd0c-5e49951a1d12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:15.131359 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3c50a8-0285-4a32-b53c-e1c672470e34 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:15.163745 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3c50a8-0285-4a32-b53c-e1c672470e34 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:15.184114 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331c917e-2233-47ef-9b0a-6be65da0f219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:15.188944 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c917e-2233-47ef-9b0a-6be65da0f219 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:15.188944 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c917e-2233-47ef-9b0a-6be65da0f219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:15.188944 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c917e-2233-47ef-9b0a-6be65da0f219 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:15.193453 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331c917e-2233-47ef-9b0a-6be65da0f219 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:15.198900 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331c917e-2233-47ef-9b0a-6be65da0f219 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:15.233151 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 957/1882] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:15 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:15.241220 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331c917e-2233-47ef-9b0a-6be65da0f219 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:15.249908 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a02f241-55ee-4cfc-bd0c-5e49951a1d12 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:15.255432 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed8b288-16db-47c4-bba7-c11379d34b14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:15.256033 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8b288-16db-47c4-bba7-c11379d34b14 service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:15.256383 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8b288-16db-47c4-bba7-c11379d34b14 service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:15.256590 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8b288-16db-47c4-bba7-c11379d34b14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:15.259672 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-2ed8b288-16db-47c4-bba7-c11379d34b14 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:15.273185 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331c917e-2233-47ef-9b0a-6be65da0f219 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:15.293749 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ea4d77-74be-40f2-a62c-06c0491a3fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:15.296715 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea4d77-74be-40f2-a62c-06c0491a3fc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:15.297025 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea4d77-74be-40f2-a62c-06c0491a3fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:15.297315 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea4d77-74be-40f2-a62c-06c0491a3fc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:15.300643 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ea4d77-74be-40f2-a62c-06c0491a3fc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:15.304245 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ea4d77-74be-40f2-a62c-06c0491a3fc1 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:15.359676 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 958/1883] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:15 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:15.364475 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ea4d77-74be-40f2-a62c-06c0491a3fc1 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:15.409503 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ea4d77-74be-40f2-a62c-06c0491a3fc1 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:15.415211 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed8b288-16db-47c4-bba7-c11379d34b14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:15.417133 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed8b288-16db-47c4-bba7-c11379d34b14 service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:15.424944 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8868392-dfe1-4d32-9d14-3f72cd91c96b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:15.425881 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8868392-dfe1-4d32-9d14-3f72cd91c96b admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:15.426222 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8868392-dfe1-4d32-9d14-3f72cd91c96b admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:15.426479 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8868392-dfe1-4d32-9d14-3f72cd91c96b admin admin] PATH_INFO: `/v3/projects/53eca8a8947b4573873d6346122c9fdb/users/577e9b59123f44d3ab3a6f21390604e5/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:15.452651 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 926/1884] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:15 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:15.475640 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8868392-dfe1-4d32-9d14-3f72cd91c96b admin admin] RBAC: Authorizing `identity:create_grant(project_id=53eca8a8947b4573873d6346122c9fdb, user_id=577e9b59123f44d3ab3a6f21390604e5, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:15.478827 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8868392-dfe1-4d32-9d14-3f72cd91c96b admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:15.508695 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 959/1885] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:15 2022] PUT /identity/v3/projects/53eca8a8947b4573873d6346122c9fdb/users/577e9b59123f44d3ab3a6f21390604e5/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:15.525802 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a366bf91-2f45-4b1f-9915-157f459352ed None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:15.526094 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a366bf91-2f45-4b1f-9915-157f459352ed None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:15.526423 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a366bf91-2f45-4b1f-9915-157f459352ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:15.605235 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-a366bf91-2f45-4b1f-9915-157f459352ed None None] MFA Rules not processed for user `577e9b59123f44d3ab3a6f21390604e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:15.606560 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-a366bf91-2f45-4b1f-9915-157f459352ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:15.811811 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 927/1886] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:15.825865 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8868392-dfe1-4d32-9d14-3f72cd91c96b admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:15.828035 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-e8868392-dfe1-4d32-9d14-3f72cd91c96b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:15.933429 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8868392-dfe1-4d32-9d14-3f72cd91c96b admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:15.938964 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa8feba-26f1-4c06-93b1-2ec42e55ad70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:15.940557 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8feba-26f1-4c06-93b1-2ec42e55ad70 service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:15.940778 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8feba-26f1-4c06-93b1-2ec42e55ad70 service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:15.941244 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8feba-26f1-4c06-93b1-2ec42e55ad70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:15.957625 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa8feba-26f1-4c06-93b1-2ec42e55ad70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:15.960439 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa8feba-26f1-4c06-93b1-2ec42e55ad70 service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:15.987143 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 960/1887] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:25.475887 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a366bf91-2f45-4b1f-9915-157f459352ed None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:25.477330 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-a366bf91-2f45-4b1f-9915-157f459352ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:25.496177 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:25.548266 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a366bf91-2f45-4b1f-9915-157f459352ed None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:25.553211 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef29e68-c40d-4349-91e1-01fd096363a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:25.554288 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef29e68-c40d-4349-91e1-01fd096363a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:25.554570 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef29e68-c40d-4349-91e1-01fd096363a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:25.554844 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef29e68-c40d-4349-91e1-01fd096363a0 admin admin] PATH_INFO: `/v3/users/872badbeaf524e6d873814e961f35f3e` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:25.556043 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef29e68-c40d-4349-91e1-01fd096363a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=872badbeaf524e6d873814e961f35f3e)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:25.558101 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef29e68-c40d-4349-91e1-01fd096363a0 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:25.618299 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-5ef29e68-c40d-4349-91e1-01fd096363a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '872badbeaf524e6d873814e961f35f3e'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:25.634841 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-5ef29e68-c40d-4349-91e1-01fd096363a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '872badbeaf524e6d873814e961f35f3e'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:25.636180 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-5ef29e68-c40d-4349-91e1-01fd096363a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '872badbeaf524e6d873814e961f35f3e'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:25.646249 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-5ef29e68-c40d-4349-91e1-01fd096363a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '872badbeaf524e6d873814e961f35f3e'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:25.647083 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-5ef29e68-c40d-4349-91e1-01fd096363a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '872badbeaf524e6d873814e961f35f3e'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:25.662008 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 928/1888] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:41:25 2022] DELETE /identity/v3/users/872badbeaf524e6d873814e961f35f3e => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:25.666736 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73fb519-5617-444f-ab37-dff1e557557b None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:25.667289 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73fb519-5617-444f-ab37-dff1e557557b None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:25.667423 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73fb519-5617-444f-ab37-dff1e557557b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:25.684620 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-c73fb519-5617-444f-ab37-dff1e557557b None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:25.685898 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-c73fb519-5617-444f-ab37-dff1e557557b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:25.752086 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 961/1889] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:25.762859 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef29e68-c40d-4349-91e1-01fd096363a0 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:25.764319 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-5ef29e68-c40d-4349-91e1-01fd096363a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:25.778254 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73fb519-5617-444f-ab37-dff1e557557b None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:25.779691 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-c73fb519-5617-444f-ab37-dff1e557557b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:25.791868 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73fb519-5617-444f-ab37-dff1e557557b None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:25.796781 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b514a-1df1-4559-b5a7-7b530ae73d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:25.799086 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b514a-1df1-4559-b5a7-7b530ae73d8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:25.799086 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b514a-1df1-4559-b5a7-7b530ae73d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:25.799086 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b514a-1df1-4559-b5a7-7b530ae73d8c admin admin] PATH_INFO: `/v3/users/a5b9c4b8ec34467a8955aff85797e9ac` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:25.800390 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471b514a-1df1-4559-b5a7-7b530ae73d8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5b9c4b8ec34467a8955aff85797e9ac)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:25.802615 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471b514a-1df1-4559-b5a7-7b530ae73d8c admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:25.860108 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef29e68-c40d-4349-91e1-01fd096363a0 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:25.867160 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6283c9b3-be7a-4f65-b653-04bee1bf4bf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:25.868163 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6283c9b3-be7a-4f65-b653-04bee1bf4bf9 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:25.868163 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6283c9b3-be7a-4f65-b653-04bee1bf4bf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:25.868964 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6283c9b3-be7a-4f65-b653-04bee1bf4bf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:25.875845 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:INSERT=4,keystone:SELECT=90,keystone:DELETE=6 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:25.885129 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-471b514a-1df1-4559-b5a7-7b530ae73d8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5b9c4b8ec34467a8955aff85797e9ac'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:25.887016 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6283c9b3-be7a-4f65-b653-04bee1bf4bf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:25.889119 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6283c9b3-be7a-4f65-b653-04bee1bf4bf9 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:25.906318 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-471b514a-1df1-4559-b5a7-7b530ae73d8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5b9c4b8ec34467a8955aff85797e9ac'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:25.907085 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 929/1890] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:25.917851 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-471b514a-1df1-4559-b5a7-7b530ae73d8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5b9c4b8ec34467a8955aff85797e9ac'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:25.919228 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-471b514a-1df1-4559-b5a7-7b530ae73d8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5b9c4b8ec34467a8955aff85797e9ac'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:25.920089 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-471b514a-1df1-4559-b5a7-7b530ae73d8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5b9c4b8ec34467a8955aff85797e9ac'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:25.939226 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 962/1891] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:41:25 2022] DELETE /identity/v3/users/a5b9c4b8ec34467a8955aff85797e9ac => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:41:25.949865 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d66a83-5c6f-436a-9176-df55ae18883f None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:25.950790 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d66a83-5c6f-436a-9176-df55ae18883f None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:25.951328 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d66a83-5c6f-436a-9176-df55ae18883f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:25.992235 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-c7d66a83-5c6f-436a-9176-df55ae18883f None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:25.994080 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-c7d66a83-5c6f-436a-9176-df55ae18883f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:26.037530 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:26.072829 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 930/1892] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:26.082795 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b514a-1df1-4559-b5a7-7b530ae73d8c admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:26.084358 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-471b514a-1df1-4559-b5a7-7b530ae73d8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:26.154661 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b514a-1df1-4559-b5a7-7b530ae73d8c admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:26.158302 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75956fcf-16bf-40df-b9fa-e9255637fb25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:26.159046 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75956fcf-16bf-40df-b9fa-e9255637fb25 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:26.159233 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75956fcf-16bf-40df-b9fa-e9255637fb25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:26.159508 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75956fcf-16bf-40df-b9fa-e9255637fb25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:26.170661 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75956fcf-16bf-40df-b9fa-e9255637fb25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:26.172138 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75956fcf-16bf-40df-b9fa-e9255637fb25 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:26.183824 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 963/1893] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:27.073613 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d66a83-5c6f-436a-9176-df55ae18883f None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:27.077156 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-c7d66a83-5c6f-436a-9176-df55ae18883f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:27.101372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d66a83-5c6f-436a-9176-df55ae18883f None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:27.107792 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9df7224-5f57-4204-974e-5ba912061564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:27.108874 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df7224-5f57-4204-974e-5ba912061564 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:27.109260 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df7224-5f57-4204-974e-5ba912061564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:27.109542 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df7224-5f57-4204-974e-5ba912061564 admin admin] PATH_INFO: `/v3/projects/096e03d26cba4d33a3a1f272d1bb00e0` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:27.110828 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9df7224-5f57-4204-974e-5ba912061564 admin admin] RBAC: Authorizing `identity:delete_project(project_id=096e03d26cba4d33a3a1f272d1bb00e0)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:27.112878 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9df7224-5f57-4204-974e-5ba912061564 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:27.193357 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f9df7224-5f57-4204-974e-5ba912061564 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '096e03d26cba4d33a3a1f272d1bb00e0', 'request_id': 'req-f9df7224-5f57-4204-974e-5ba912061564'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:27.201985 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f9df7224-5f57-4204-974e-5ba912061564 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '096e03d26cba4d33a3a1f272d1bb00e0', 'request_id': 'req-f9df7224-5f57-4204-974e-5ba912061564'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:27.214316 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f9df7224-5f57-4204-974e-5ba912061564 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '096e03d26cba4d33a3a1f272d1bb00e0', 'request_id': 'req-f9df7224-5f57-4204-974e-5ba912061564'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:27.230658 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f9df7224-5f57-4204-974e-5ba912061564 admin admin] The token cache is being invalidate because project 096e03d26cba4d33a3a1f272d1bb00e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:41:27.230910 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f9df7224-5f57-4204-974e-5ba912061564 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:27.231593 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f9df7224-5f57-4204-974e-5ba912061564 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:27.233550 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 931/1894] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:41:27 2022] DELETE /identity/v3/projects/096e03d26cba4d33a3a1f272d1bb00e0 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:27.235242 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75956fcf-16bf-40df-b9fa-e9255637fb25 service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:27.236770 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-75956fcf-16bf-40df-b9fa-e9255637fb25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:27.237029 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9df7224-5f57-4204-974e-5ba912061564 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:27.239279 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-f9df7224-5f57-4204-974e-5ba912061564 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:27.254009 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75956fcf-16bf-40df-b9fa-e9255637fb25 service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:27.333733 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9df7224-5f57-4204-974e-5ba912061564 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:27.334448 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e5b349-cb29-4411-9319-95b40a3f4398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:27.335496 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e5b349-cb29-4411-9319-95b40a3f4398 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:27.336994 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e5b349-cb29-4411-9319-95b40a3f4398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:27.337467 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e5b349-cb29-4411-9319-95b40a3f4398 admin admin] PATH_INFO: `/v3/projects/e6b2d4882c634c8f813aa423ab719fe5` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:27.338581 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8185663d-cb65-4495-b6d5-b76335aef787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:27.338722 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e5b349-cb29-4411-9319-95b40a3f4398 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6b2d4882c634c8f813aa423ab719fe5)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:27.339731 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8185663d-cb65-4495-b6d5-b76335aef787 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:27.340017 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8185663d-cb65-4495-b6d5-b76335aef787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:27.340289 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8185663d-cb65-4495-b6d5-b76335aef787 admin admin] PATH_INFO: `/v3/users/51c1f9409afb42ac93952aca941916b3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:27.341449 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e5b349-cb29-4411-9319-95b40a3f4398 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:27.341622 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8185663d-cb65-4495-b6d5-b76335aef787 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51c1f9409afb42ac93952aca941916b3)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:27.344296 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8185663d-cb65-4495-b6d5-b76335aef787 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:27.408718 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-8185663d-cb65-4495-b6d5-b76335aef787 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51c1f9409afb42ac93952aca941916b3'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:27.421224 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:27.422653 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-8185663d-cb65-4495-b6d5-b76335aef787 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51c1f9409afb42ac93952aca941916b3'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:27.423597 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-8185663d-cb65-4495-b6d5-b76335aef787 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51c1f9409afb42ac93952aca941916b3'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:27.424688 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c3e5b349-cb29-4411-9319-95b40a3f4398 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6b2d4882c634c8f813aa423ab719fe5', 'request_id': 'req-c3e5b349-cb29-4411-9319-95b40a3f4398'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:27.434465 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c3e5b349-cb29-4411-9319-95b40a3f4398 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6b2d4882c634c8f813aa423ab719fe5', 'request_id': 'req-c3e5b349-cb29-4411-9319-95b40a3f4398'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:27.441041 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c3e5b349-cb29-4411-9319-95b40a3f4398 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6b2d4882c634c8f813aa423ab719fe5', 'request_id': 'req-c3e5b349-cb29-4411-9319-95b40a3f4398'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:27.444045 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:27.448038 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-8185663d-cb65-4495-b6d5-b76335aef787 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51c1f9409afb42ac93952aca941916b3'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:27.449248 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-8185663d-cb65-4495-b6d5-b76335aef787 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51c1f9409afb42ac93952aca941916b3'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:27.465573 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c3e5b349-cb29-4411-9319-95b40a3f4398 admin admin] The token cache is being invalidate because project e6b2d4882c634c8f813aa423ab719fe5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:41:27.465949 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c3e5b349-cb29-4411-9319-95b40a3f4398 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:27.468163 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c3e5b349-cb29-4411-9319-95b40a3f4398 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:27.468583 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 932/1895] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:41:27 2022] DELETE /identity/v3/users/51c1f9409afb42ac93952aca941916b3 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:41:27.470172 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c46afd-cb60-4a24-ba4d-b8bf88019c75 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:27.470464 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c46afd-cb60-4a24-ba4d-b8bf88019c75 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:27.470741 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c46afd-cb60-4a24-ba4d-b8bf88019c75 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:27.477120 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 964/1896] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:41:27 2022] DELETE /identity/v3/projects/e6b2d4882c634c8f813aa423ab719fe5 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:27.478397 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e5b349-cb29-4411-9319-95b40a3f4398 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:27.480939 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-c3e5b349-cb29-4411-9319-95b40a3f4398 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:27.491191 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 933/1897] 10.222.0.13 () {54 vars in 921 bytes} [Fri Jun 24 08:41:27 2022] GET /identity => generated 270 bytes in 22 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:41:27.500960 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39635062-1369-4d4c-ac6e-fb81fc7df3a8 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:27.501250 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39635062-1369-4d4c-ac6e-fb81fc7df3a8 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:27.501537 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39635062-1369-4d4c-ac6e-fb81fc7df3a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:27.531501 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-39635062-1369-4d4c-ac6e-fb81fc7df3a8 None None] MFA Rules not processed for user `74bb1bc6821b4500b26f160ccae5b7d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:27.532747 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-39635062-1369-4d4c-ac6e-fb81fc7df3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:27.569385 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e5b349-cb29-4411-9319-95b40a3f4398 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:27.658253 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b195d0bb-ca7e-43e5-8214-ac26b34ecf2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:27.660974 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195d0bb-ca7e-43e5-8214-ac26b34ecf2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:27.661718 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195d0bb-ca7e-43e5-8214-ac26b34ecf2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:27.663003 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195d0bb-ca7e-43e5-8214-ac26b34ecf2a admin admin] PATH_INFO: `/v3/users/359474792f2548a2b2b73c88898d9abb` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:27.665665 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b195d0bb-ca7e-43e5-8214-ac26b34ecf2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=359474792f2548a2b2b73c88898d9abb)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:27.668241 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b195d0bb-ca7e-43e5-8214-ac26b34ecf2a admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:27.679611 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 934/1898] 10.222.0.13 () {58 vars in 1018 bytes} [Fri Jun 24 08:41:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:27.731894 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-b195d0bb-ca7e-43e5-8214-ac26b34ecf2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '359474792f2548a2b2b73c88898d9abb'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:27.748662 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-b195d0bb-ca7e-43e5-8214-ac26b34ecf2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '359474792f2548a2b2b73c88898d9abb'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:27.772191 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-b195d0bb-ca7e-43e5-8214-ac26b34ecf2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '359474792f2548a2b2b73c88898d9abb'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:27.772836 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-b195d0bb-ca7e-43e5-8214-ac26b34ecf2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '359474792f2548a2b2b73c88898d9abb'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:27.773584 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-b195d0bb-ca7e-43e5-8214-ac26b34ecf2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '359474792f2548a2b2b73c88898d9abb'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:27.781752 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39635062-1369-4d4c-ac6e-fb81fc7df3a8 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:27.784850 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-39635062-1369-4d4c-ac6e-fb81fc7df3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:27.799878 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39635062-1369-4d4c-ac6e-fb81fc7df3a8 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:27.801635 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 965/1899] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:41:27 2022] DELETE /identity/v3/users/359474792f2548a2b2b73c88898d9abb => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:27.805609 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b195d0bb-ca7e-43e5-8214-ac26b34ecf2a admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:27.807127 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b195d0bb-ca7e-43e5-8214-ac26b34ecf2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:27.831561 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:27.895208 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a33c099-baab-4336-ba30-8fd94958d05b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74bb1bc6821b4500b26f160ccae5b7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:27.896175 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a33c099-baab-4336-ba30-8fd94958d05b service nova] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:27.896546 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a33c099-baab-4336-ba30-8fd94958d05b service nova] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:27.897262 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a33c099-baab-4336-ba30-8fd94958d05b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:27.899308 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-5a33c099-baab-4336-ba30-8fd94958d05b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:27.914745 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b195d0bb-ca7e-43e5-8214-ac26b34ecf2a admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:27.914876 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a33c099-baab-4336-ba30-8fd94958d05b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:27.918248 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a33c099-baab-4336-ba30-8fd94958d05b service nova] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:27.937984 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d4ca71-152f-4100-9bd9-f5c44f907cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:27.938404 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 935/1900] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:27.939694 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d4ca71-152f-4100-9bd9-f5c44f907cb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:27.939978 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d4ca71-152f-4100-9bd9-f5c44f907cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:27.940279 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d4ca71-152f-4100-9bd9-f5c44f907cb4 admin admin] PATH_INFO: `/v3/users/577e9b59123f44d3ab3a6f21390604e5` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:27.942734 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d4ca71-152f-4100-9bd9-f5c44f907cb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=577e9b59123f44d3ab3a6f21390604e5)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:27.945331 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d4ca71-152f-4100-9bd9-f5c44f907cb4 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:28.019515 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d7d4ca71-152f-4100-9bd9-f5c44f907cb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '577e9b59123f44d3ab3a6f21390604e5'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:28.037016 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d7d4ca71-152f-4100-9bd9-f5c44f907cb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '577e9b59123f44d3ab3a6f21390604e5'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:28.050143 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d7d4ca71-152f-4100-9bd9-f5c44f907cb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '577e9b59123f44d3ab3a6f21390604e5'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:28.051732 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d7d4ca71-152f-4100-9bd9-f5c44f907cb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '577e9b59123f44d3ab3a6f21390604e5'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:28.052576 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d7d4ca71-152f-4100-9bd9-f5c44f907cb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '577e9b59123f44d3ab3a6f21390604e5'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:28.073436 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 966/1901] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:41:27 2022] DELETE /identity/v3/users/577e9b59123f44d3ab3a6f21390604e5 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:28.077827 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df54d4f-22f7-4d50-9f17-6e8b2f36da5a None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:28.078347 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df54d4f-22f7-4d50-9f17-6e8b2f36da5a None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:28.078439 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df54d4f-22f7-4d50-9f17-6e8b2f36da5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:28.098043 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-2df54d4f-22f7-4d50-9f17-6e8b2f36da5a None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:28.099300 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-2df54d4f-22f7-4d50-9f17-6e8b2f36da5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:28.211914 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:28.218797 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d4ca71-152f-4100-9bd9-f5c44f907cb4 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:28.220271 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-d7d4ca71-152f-4100-9bd9-f5c44f907cb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:28.241217 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 936/1902] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:28.312605 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df54d4f-22f7-4d50-9f17-6e8b2f36da5a None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:28.314228 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-2df54d4f-22f7-4d50-9f17-6e8b2f36da5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:28.351625 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d4ca71-152f-4100-9bd9-f5c44f907cb4 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:28.375156 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21030817-4e8d-4cb4-9319-3bd3559e72d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:28.377619 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21030817-4e8d-4cb4-9319-3bd3559e72d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:28.378157 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21030817-4e8d-4cb4-9319-3bd3559e72d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:28.378468 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21030817-4e8d-4cb4-9319-3bd3559e72d2 admin admin] PATH_INFO: `/v3/projects/ffa71ae1020e41fba96f2f5b66c0b91e` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:28.381020 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21030817-4e8d-4cb4-9319-3bd3559e72d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffa71ae1020e41fba96f2f5b66c0b91e)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:28.383107 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21030817-4e8d-4cb4-9319-3bd3559e72d2 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:28.448134 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:28.454450 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df54d4f-22f7-4d50-9f17-6e8b2f36da5a None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:28.460585 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-21030817-4e8d-4cb4-9319-3bd3559e72d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffa71ae1020e41fba96f2f5b66c0b91e', 'request_id': 'req-21030817-4e8d-4cb4-9319-3bd3559e72d2'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:28.474353 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-21030817-4e8d-4cb4-9319-3bd3559e72d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffa71ae1020e41fba96f2f5b66c0b91e', 'request_id': 'req-21030817-4e8d-4cb4-9319-3bd3559e72d2'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:28.483576 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-21030817-4e8d-4cb4-9319-3bd3559e72d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffa71ae1020e41fba96f2f5b66c0b91e', 'request_id': 'req-21030817-4e8d-4cb4-9319-3bd3559e72d2'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:28.524701 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375b1441-04e5-42e4-825d-17388abef3e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:28.525274 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-21030817-4e8d-4cb4-9319-3bd3559e72d2 admin admin] The token cache is being invalidate because project ffa71ae1020e41fba96f2f5b66c0b91e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:41:28.525572 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-21030817-4e8d-4cb4-9319-3bd3559e72d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:28.526371 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b1441-04e5-42e4-825d-17388abef3e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:28.526810 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b1441-04e5-42e4-825d-17388abef3e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:28.527232 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b1441-04e5-42e4-825d-17388abef3e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:28.529727 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-375b1441-04e5-42e4-825d-17388abef3e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:28.531742 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-21030817-4e8d-4cb4-9319-3bd3559e72d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:28.535406 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 967/1903] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:41:28 2022] DELETE /identity/v3/projects/ffa71ae1020e41fba96f2f5b66c0b91e => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:28.541789 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21030817-4e8d-4cb4-9319-3bd3559e72d2 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:28.546970 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-21030817-4e8d-4cb4-9319-3bd3559e72d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:28.635054 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375b1441-04e5-42e4-825d-17388abef3e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:28.636607 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375b1441-04e5-42e4-825d-17388abef3e5 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:28.651416 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 937/1904] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:41:28.653755 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67483c1a-0d07-4de2-8274-32bffe5af527 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:28.654141 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67483c1a-0d07-4de2-8274-32bffe5af527 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:28.654495 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67483c1a-0d07-4de2-8274-32bffe5af527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:28.682732 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-67483c1a-0d07-4de2-8274-32bffe5af527 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:28.684607 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-67483c1a-0d07-4de2-8274-32bffe5af527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:28.689787 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21030817-4e8d-4cb4-9319-3bd3559e72d2 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:28.696684 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 938/1905] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:28.698633 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e742ee9-1f6e-4c4b-9fc3-d6a6690e86e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:28.699631 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e742ee9-1f6e-4c4b-9fc3-d6a6690e86e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:28.699914 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e742ee9-1f6e-4c4b-9fc3-d6a6690e86e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:28.700193 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e742ee9-1f6e-4c4b-9fc3-d6a6690e86e2 admin admin] PATH_INFO: `/v3/projects/1ef8bdc250be4dbfabceb417a73bc2ec` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:28.701573 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e742ee9-1f6e-4c4b-9fc3-d6a6690e86e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ef8bdc250be4dbfabceb417a73bc2ec)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:28.704164 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e742ee9-1f6e-4c4b-9fc3-d6a6690e86e2 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:28.706035 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84181551-a9a7-4e31-8cce-7e39efbe32d4 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:28.706891 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84181551-a9a7-4e31-8cce-7e39efbe32d4 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:28.707469 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84181551-a9a7-4e31-8cce-7e39efbe32d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:28.743329 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-84181551-a9a7-4e31-8cce-7e39efbe32d4 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:28.744577 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-84181551-a9a7-4e31-8cce-7e39efbe32d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:28.760580 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 939/1906] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:28.762024 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84181551-a9a7-4e31-8cce-7e39efbe32d4 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:28.775774 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84181551-a9a7-4e31-8cce-7e39efbe32d4 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:28.788056 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-3e742ee9-1f6e-4c4b-9fc3-d6a6690e86e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ef8bdc250be4dbfabceb417a73bc2ec', 'request_id': 'req-3e742ee9-1f6e-4c4b-9fc3-d6a6690e86e2'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:28.799668 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-3e742ee9-1f6e-4c4b-9fc3-d6a6690e86e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ef8bdc250be4dbfabceb417a73bc2ec', 'request_id': 'req-3e742ee9-1f6e-4c4b-9fc3-d6a6690e86e2'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:28.808696 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-3e742ee9-1f6e-4c4b-9fc3-d6a6690e86e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ef8bdc250be4dbfabceb417a73bc2ec', 'request_id': 'req-3e742ee9-1f6e-4c4b-9fc3-d6a6690e86e2'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:28.842418 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-3e742ee9-1f6e-4c4b-9fc3-d6a6690e86e2 admin admin] The token cache is being invalidate because project 1ef8bdc250be4dbfabceb417a73bc2ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:41:28.842718 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-3e742ee9-1f6e-4c4b-9fc3-d6a6690e86e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:28.843214 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:28.843776 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-3e742ee9-1f6e-4c4b-9fc3-d6a6690e86e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:28.846275 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 968/1907] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:41:28 2022] DELETE /identity/v3/projects/1ef8bdc250be4dbfabceb417a73bc2ec => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:28.856175 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc16eda-f8e8-452d-ae14-9e58366a5ce3 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:28.857779 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc16eda-f8e8-452d-ae14-9e58366a5ce3 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:28.857887 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476e4d64-fe13-4c92-adf8-99f87c8eabec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:28.857996 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc16eda-f8e8-452d-ae14-9e58366a5ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:28.858081 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e4d64-fe13-4c92-adf8-99f87c8eabec admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:28.858645 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e4d64-fe13-4c92-adf8-99f87c8eabec admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:28.858988 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e4d64-fe13-4c92-adf8-99f87c8eabec admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:28.859872 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476e4d64-fe13-4c92-adf8-99f87c8eabec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:41:28.860246 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476e4d64-fe13-4c92-adf8-99f87c8eabec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:28.862448 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476e4d64-fe13-4c92-adf8-99f87c8eabec admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:28.883038 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 940/1908] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:41:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:41:28.884377 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476e4d64-fe13-4c92-adf8-99f87c8eabec admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:28.886660 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-476e4d64-fe13-4c92-adf8-99f87c8eabec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:28.901310 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-ddc16eda-f8e8-452d-ae14-9e58366a5ce3 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:28.902882 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-ddc16eda-f8e8-452d-ae14-9e58366a5ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:28.916239 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 969/1909] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:28.917225 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476e4d64-fe13-4c92-adf8-99f87c8eabec admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:28.925374 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472fafc3-7831-452b-85a9-fc7d56f06941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:28.925374 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472fafc3-7831-452b-85a9-fc7d56f06941 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:28.925374 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472fafc3-7831-452b-85a9-fc7d56f06941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:28.925810 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472fafc3-7831-452b-85a9-fc7d56f06941 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:28.926894 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472fafc3-7831-452b-85a9-fc7d56f06941 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:28.929219 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472fafc3-7831-452b-85a9-fc7d56f06941 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:28.931097 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32230d-313b-4c08-8555-c1a397cf167f None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:28.931596 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32230d-313b-4c08-8555-c1a397cf167f None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:28.932055 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32230d-313b-4c08-8555-c1a397cf167f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:28.959148 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-2e32230d-313b-4c08-8555-c1a397cf167f None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:28.960948 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-2e32230d-313b-4c08-8555-c1a397cf167f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:28.973013 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 970/1910] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:28.979188 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 941/1911] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:41:28 2022] POST /identity/v3/projects => generated 379 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:28.983177 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e32230d-313b-4c08-8555-c1a397cf167f None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:28.989342 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472fafc3-7831-452b-85a9-fc7d56f06941 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:29.011098 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e32230d-313b-4c08-8555-c1a397cf167f None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:29.133790 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:29.158588 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472fafc3-7831-452b-85a9-fc7d56f06941 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:29.166242 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8628ccd4-554c-4875-be1d-41fe5c735349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:29.167100 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89587834-1ad3-43e5-84ee-00e2a72d2696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:29.167247 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8628ccd4-554c-4875-be1d-41fe5c735349 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:29.167247 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8628ccd4-554c-4875-be1d-41fe5c735349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:29.167397 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89587834-1ad3-43e5-84ee-00e2a72d2696 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:29.167469 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8628ccd4-554c-4875-be1d-41fe5c735349 admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:29.167680 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89587834-1ad3-43e5-84ee-00e2a72d2696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:29.168036 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89587834-1ad3-43e5-84ee-00e2a72d2696 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:29.168581 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8628ccd4-554c-4875-be1d-41fe5c735349 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:41:29.168884 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89587834-1ad3-43e5-84ee-00e2a72d2696 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:29.168984 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8628ccd4-554c-4875-be1d-41fe5c735349 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:29.170572 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8628ccd4-554c-4875-be1d-41fe5c735349 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:29.171875 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89587834-1ad3-43e5-84ee-00e2a72d2696 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:29.185123 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 971/1912] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:41:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:41:29.190223 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8628ccd4-554c-4875-be1d-41fe5c735349 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:29.204931 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8628ccd4-554c-4875-be1d-41fe5c735349 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:29.206462 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 942/1913] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:41:28 2022] POST /identity/v3/users => generated 290 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:29.210865 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2985f7-1fa0-421b-ae3a-5b42d0a108dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:29.211161 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89587834-1ad3-43e5-84ee-00e2a72d2696 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:29.211461 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2985f7-1fa0-421b-ae3a-5b42d0a108dc admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:29.211748 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2985f7-1fa0-421b-ae3a-5b42d0a108dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:29.212056 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2985f7-1fa0-421b-ae3a-5b42d0a108dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:29.212813 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2985f7-1fa0-421b-ae3a-5b42d0a108dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:29.214842 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2985f7-1fa0-421b-ae3a-5b42d0a108dc admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:29.249930 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89587834-1ad3-43e5-84ee-00e2a72d2696 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:29.267956 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1c6648-3a0b-4631-80b6-3d4a003f949e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:29.270318 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1c6648-3a0b-4631-80b6-3d4a003f949e admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:29.270980 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 972/1914] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:41:29 2022] POST /identity/v3/projects => generated 369 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:29.274650 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1c6648-3a0b-4631-80b6-3d4a003f949e admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:29.275030 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1c6648-3a0b-4631-80b6-3d4a003f949e admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:29.276698 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1c6648-3a0b-4631-80b6-3d4a003f949e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:29.278508 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1c6648-3a0b-4631-80b6-3d4a003f949e admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:29.279088 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2985f7-1fa0-421b-ae3a-5b42d0a108dc admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:29.303119 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 943/1915] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:29 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:41:29.307418 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1c6648-3a0b-4631-80b6-3d4a003f949e admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:29.324854 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1c6648-3a0b-4631-80b6-3d4a003f949e admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:29.383293 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2985f7-1fa0-421b-ae3a-5b42d0a108dc admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:29.389141 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17138a9-aec3-44f2-ac98-4bdc0c640dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:29.389938 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17138a9-aec3-44f2-ac98-4bdc0c640dc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:29.390157 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17138a9-aec3-44f2-ac98-4bdc0c640dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:29.390440 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17138a9-aec3-44f2-ac98-4bdc0c640dc3 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:29.391149 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17138a9-aec3-44f2-ac98-4bdc0c640dc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:29.394083 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17138a9-aec3-44f2-ac98-4bdc0c640dc3 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:29.429625 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 973/1916] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:41:29 2022] POST /identity/v3/users => generated 285 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:29.433412 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17138a9-aec3-44f2-ac98-4bdc0c640dc3 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:29.443938 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f443291c-fe33-494f-9e37-0302851ba648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:29.444638 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443291c-fe33-494f-9e37-0302851ba648 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:29.446066 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443291c-fe33-494f-9e37-0302851ba648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:29.446567 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443291c-fe33-494f-9e37-0302851ba648 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:29.447536 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f443291c-fe33-494f-9e37-0302851ba648 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:29.450036 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f443291c-fe33-494f-9e37-0302851ba648 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:29.454442 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17138a9-aec3-44f2-ac98-4bdc0c640dc3 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:29.460683 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcebe3f-cf63-4079-a871-2a8a6a80b0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:29.461449 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcebe3f-cf63-4079-a871-2a8a6a80b0fb admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:29.461670 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcebe3f-cf63-4079-a871-2a8a6a80b0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:29.461983 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcebe3f-cf63-4079-a871-2a8a6a80b0fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:29.462998 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcebe3f-cf63-4079-a871-2a8a6a80b0fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:29.465064 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcebe3f-cf63-4079-a871-2a8a6a80b0fb admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:29.467596 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 944/1917] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:29 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:29.472498 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f443291c-fe33-494f-9e37-0302851ba648 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:29.481830 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 974/1918] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:29 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:41:29.485869 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcebe3f-cf63-4079-a871-2a8a6a80b0fb admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:29.492292 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f443291c-fe33-494f-9e37-0302851ba648 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:29.498341 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4422b1a0-6ee8-4b16-87d5-7e907800057e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:29.510209 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcebe3f-cf63-4079-a871-2a8a6a80b0fb admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:29.510209 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ffc60-cfe3-457c-9c48-e560a2529d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:29.510209 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ffc60-cfe3-457c-9c48-e560a2529d3f admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:29.510209 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ffc60-cfe3-457c-9c48-e560a2529d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:29.510566 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ffc60-cfe3-457c-9c48-e560a2529d3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:29.510566 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2ffc60-cfe3-457c-9c48-e560a2529d3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:29.510566 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2ffc60-cfe3-457c-9c48-e560a2529d3f admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:29.513236 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4422b1a0-6ee8-4b16-87d5-7e907800057e admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:29.513533 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4422b1a0-6ee8-4b16-87d5-7e907800057e admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:29.513847 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4422b1a0-6ee8-4b16-87d5-7e907800057e admin admin] PATH_INFO: `/v3/projects/82c77153f8b14e028c543cf16a408c85/users/b099108612094ff5b0d20e63b4259fa8/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:29.524985 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 975/1919] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:29 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:29.530740 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ffc60-cfe3-457c-9c48-e560a2529d3f admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:29.545137 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4422b1a0-6ee8-4b16-87d5-7e907800057e admin admin] RBAC: Authorizing `identity:create_grant(project_id=82c77153f8b14e028c543cf16a408c85, user_id=b099108612094ff5b0d20e63b4259fa8, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:29.547578 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ffc60-cfe3-457c-9c48-e560a2529d3f admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:29.549017 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4422b1a0-6ee8-4b16-87d5-7e907800057e admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:29.554727 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444ebf8c-0852-4806-8f2c-ed0700af7d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:29.555802 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444ebf8c-0852-4806-8f2c-ed0700af7d65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:29.556151 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444ebf8c-0852-4806-8f2c-ed0700af7d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:29.556435 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444ebf8c-0852-4806-8f2c-ed0700af7d65 admin admin] PATH_INFO: `/v3/projects/d5f00d9c9e94486fbaf2297bab2fcad3/users/302f4ed3726d489b9dcd19969fc23c69/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:29.571202 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 945/1920] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:29 2022] PUT /identity/v3/projects/82c77153f8b14e028c543cf16a408c85/users/b099108612094ff5b0d20e63b4259fa8/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:29.583817 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dea930-4fbe-4982-a190-22b722604cc8 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:29.584125 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dea930-4fbe-4982-a190-22b722604cc8 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:29.584417 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dea930-4fbe-4982-a190-22b722604cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:29.586263 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444ebf8c-0852-4806-8f2c-ed0700af7d65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5f00d9c9e94486fbaf2297bab2fcad3, user_id=302f4ed3726d489b9dcd19969fc23c69, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:29.590108 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444ebf8c-0852-4806-8f2c-ed0700af7d65 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:29.614045 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:29.617966 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 976/1921] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:29 2022] PUT /identity/v3/projects/d5f00d9c9e94486fbaf2297bab2fcad3/users/302f4ed3726d489b9dcd19969fc23c69/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:29.630420 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe55f08-98b5-46a0-bc02-20b1cc1b490e None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:29.630732 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe55f08-98b5-46a0-bc02-20b1cc1b490e None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:29.631014 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe55f08-98b5-46a0-bc02-20b1cc1b490e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:29.633619 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-41dea930-4fbe-4982-a190-22b722604cc8 None None] MFA Rules not processed for user `b099108612094ff5b0d20e63b4259fa8`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:29.635812 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-41dea930-4fbe-4982-a190-22b722604cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:29.668623 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-9fe55f08-98b5-46a0-bc02-20b1cc1b490e None None] MFA Rules not processed for user `302f4ed3726d489b9dcd19969fc23c69`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:29.670869 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-9fe55f08-98b5-46a0-bc02-20b1cc1b490e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:29.729011 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:29.756574 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 977/1922] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:29 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:29.757656 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe55f08-98b5-46a0-bc02-20b1cc1b490e None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:29.759269 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-9fe55f08-98b5-46a0-bc02-20b1cc1b490e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:29.764324 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 946/1923] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:29 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:29.765844 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dea930-4fbe-4982-a190-22b722604cc8 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:29.840119 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe55f08-98b5-46a0-bc02-20b1cc1b490e None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:29.846991 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b7c093-4219-4b4b-aac3-d16aa14ee40a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:29.848884 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7c093-4219-4b4b-aac3-d16aa14ee40a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:29.849184 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7c093-4219-4b4b-aac3-d16aa14ee40a admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:29.849506 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7c093-4219-4b4b-aac3-d16aa14ee40a admin admin] PATH_INFO: `/v3/projects/53eca8a8947b4573873d6346122c9fdb` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:29.850994 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b7c093-4219-4b4b-aac3-d16aa14ee40a admin admin] RBAC: Authorizing `identity:delete_project(project_id=53eca8a8947b4573873d6346122c9fdb)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:29.852324 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dea930-4fbe-4982-a190-22b722604cc8 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:29.854478 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b7c093-4219-4b4b-aac3-d16aa14ee40a admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:29.857849 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac30638-e86a-4b77-85e0-a7b478cd985d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:29.858521 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac30638-e86a-4b77-85e0-a7b478cd985d admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:29.858807 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac30638-e86a-4b77-85e0-a7b478cd985d admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:29.859080 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac30638-e86a-4b77-85e0-a7b478cd985d admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:29.859794 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac30638-e86a-4b77-85e0-a7b478cd985d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:29.861860 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac30638-e86a-4b77-85e0-a7b478cd985d admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:29.894337 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 947/1924] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:41:29 2022] POST /identity/v3/projects => generated 369 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:29.894337 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac30638-e86a-4b77-85e0-a7b478cd985d admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:29.916878 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-b8b7c093-4219-4b4b-aac3-d16aa14ee40a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53eca8a8947b4573873d6346122c9fdb', 'request_id': 'req-b8b7c093-4219-4b4b-aac3-d16aa14ee40a'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:29.923674 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-b8b7c093-4219-4b4b-aac3-d16aa14ee40a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53eca8a8947b4573873d6346122c9fdb', 'request_id': 'req-b8b7c093-4219-4b4b-aac3-d16aa14ee40a'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:29.930860 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-b8b7c093-4219-4b4b-aac3-d16aa14ee40a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53eca8a8947b4573873d6346122c9fdb', 'request_id': 'req-b8b7c093-4219-4b4b-aac3-d16aa14ee40a'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:29.945071 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-b8b7c093-4219-4b4b-aac3-d16aa14ee40a admin admin] The token cache is being invalidate because project 53eca8a8947b4573873d6346122c9fdb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:41:29.945367 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-b8b7c093-4219-4b4b-aac3-d16aa14ee40a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:29.946051 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-b8b7c093-4219-4b4b-aac3-d16aa14ee40a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:29.947912 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 978/1925] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:41:29 2022] DELETE /identity/v3/projects/53eca8a8947b4573873d6346122c9fdb => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:29.948740 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b7c093-4219-4b4b-aac3-d16aa14ee40a admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:29.949988 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b8b7c093-4219-4b4b-aac3-d16aa14ee40a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:29.963677 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac30638-e86a-4b77-85e0-a7b478cd985d admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:30.003699 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:30.025522 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b7c093-4219-4b4b-aac3-d16aa14ee40a admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:30.032031 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d041094-cdbc-4767-ba7f-c78f41e9b908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:30.033014 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d041094-cdbc-4767-ba7f-c78f41e9b908 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:30.033340 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d041094-cdbc-4767-ba7f-c78f41e9b908 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:30.033683 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d041094-cdbc-4767-ba7f-c78f41e9b908 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:30.034408 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d041094-cdbc-4767-ba7f-c78f41e9b908 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:30.037251 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d041094-cdbc-4767-ba7f-c78f41e9b908 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:30.041367 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8f9753-c8f6-462e-abc5-6e9aa58b354a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:30.042422 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f9753-c8f6-462e-abc5-6e9aa58b354a admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:30.042891 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f9753-c8f6-462e-abc5-6e9aa58b354a admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:30.043431 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f9753-c8f6-462e-abc5-6e9aa58b354a admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:30.044822 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8f9753-c8f6-462e-abc5-6e9aa58b354a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:30.049743 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8f9753-c8f6-462e-abc5-6e9aa58b354a admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:30.065866 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 979/1926] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:41:29 2022] POST /identity/v3/users => generated 291 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:30.068179 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac44ad-73d3-4731-a1e0-9f0bbede8aa6 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:30.070773 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac44ad-73d3-4731-a1e0-9f0bbede8aa6 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:30.071140 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac44ad-73d3-4731-a1e0-9f0bbede8aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:30.090167 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 948/1927] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:41:29 2022] POST /identity/v3/projects => generated 379 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:30.090973 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8f9753-c8f6-462e-abc5-6e9aa58b354a admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:30.099894 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-8aac44ad-73d3-4731-a1e0-9f0bbede8aa6 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:30.100829 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-8aac44ad-73d3-4731-a1e0-9f0bbede8aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:30.106443 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:30.167661 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8f9753-c8f6-462e-abc5-6e9aa58b354a admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:30.173445 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9223a11-8f04-4d00-b25b-19bec0dd303c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:30.174498 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9223a11-8f04-4d00-b25b-19bec0dd303c admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:30.174928 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9223a11-8f04-4d00-b25b-19bec0dd303c admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:30.175988 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9223a11-8f04-4d00-b25b-19bec0dd303c admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:30.177027 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9223a11-8f04-4d00-b25b-19bec0dd303c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:30.179184 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9223a11-8f04-4d00-b25b-19bec0dd303c admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:30.185148 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 980/1928] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:30.186318 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aac44ad-73d3-4731-a1e0-9f0bbede8aa6 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:30.187766 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-8aac44ad-73d3-4731-a1e0-9f0bbede8aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:30.192801 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 949/1929] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:30 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:30.195166 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3491b919-a93e-475e-965b-029b27f03acd None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:30.195628 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3491b919-a93e-475e-965b-029b27f03acd None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:30.196064 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3491b919-a93e-475e-965b-029b27f03acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:30.202291 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aac44ad-73d3-4731-a1e0-9f0bbede8aa6 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:30.209161 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed707ed-e824-4302-8645-0b9fb1199f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:30.210029 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed707ed-e824-4302-8645-0b9fb1199f35 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:30.210384 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed707ed-e824-4302-8645-0b9fb1199f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:30.210732 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed707ed-e824-4302-8645-0b9fb1199f35 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:30.212021 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed707ed-e824-4302-8645-0b9fb1199f35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:30.214328 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed707ed-e824-4302-8645-0b9fb1199f35 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:30.217507 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-3491b919-a93e-475e-965b-029b27f03acd None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:30.219251 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-3491b919-a93e-475e-965b-029b27f03acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:30.231088 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 950/1930] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:41:30.232410 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3491b919-a93e-475e-965b-029b27f03acd None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:30.242626 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 981/1931] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:41:30 2022] POST /identity/v3/users => generated 296 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:30.243905 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed707ed-e824-4302-8645-0b9fb1199f35 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:30.244360 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3491b919-a93e-475e-965b-029b27f03acd None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:30.250112 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed74fc0-427e-4e9f-978b-e28e1a8ee3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:30.251018 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed74fc0-427e-4e9f-978b-e28e1a8ee3e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:30.251301 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed74fc0-427e-4e9f-978b-e28e1a8ee3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:30.251577 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed74fc0-427e-4e9f-978b-e28e1a8ee3e8 admin admin] PATH_INFO: `/v3/projects/7f4018df46b041a2840da0a640e5c08e/users/16c03169f25844a2b1ef782b90aacf80/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:30.259084 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed707ed-e824-4302-8645-0b9fb1199f35 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:30.264493 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1301be90-f220-4ca4-b5aa-e09a0521920c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:30.266168 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301be90-f220-4ca4-b5aa-e09a0521920c admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:30.266168 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301be90-f220-4ca4-b5aa-e09a0521920c admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:30.266168 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301be90-f220-4ca4-b5aa-e09a0521920c admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:30.266857 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1301be90-f220-4ca4-b5aa-e09a0521920c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:41:30.267165 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1301be90-f220-4ca4-b5aa-e09a0521920c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:30.270447 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed74fc0-427e-4e9f-978b-e28e1a8ee3e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f4018df46b041a2840da0a640e5c08e, user_id=16c03169f25844a2b1ef782b90aacf80, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:30.271529 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1301be90-f220-4ca4-b5aa-e09a0521920c admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:30.277056 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed74fc0-427e-4e9f-978b-e28e1a8ee3e8 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:30.294106 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 982/1932] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:41:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:41:30.294106 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1301be90-f220-4ca4-b5aa-e09a0521920c admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:30.306374 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 951/1933] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:30 2022] PUT /identity/v3/projects/7f4018df46b041a2840da0a640e5c08e/users/16c03169f25844a2b1ef782b90aacf80/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:30.307187 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed74fc0-427e-4e9f-978b-e28e1a8ee3e8 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:30.315205 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1301be90-f220-4ca4-b5aa-e09a0521920c admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:30.376517 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1be2601-0eea-47c0-a170-c96bbbeb6949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:30.377283 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be2601-0eea-47c0-a170-c96bbbeb6949 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:30.377603 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be2601-0eea-47c0-a170-c96bbbeb6949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:30.377943 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be2601-0eea-47c0-a170-c96bbbeb6949 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:30.378959 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed74fc0-427e-4e9f-978b-e28e1a8ee3e8 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:30.379745 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1be2601-0eea-47c0-a170-c96bbbeb6949 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:30.381807 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1be2601-0eea-47c0-a170-c96bbbeb6949 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:30.388177 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36375590-3e87-47d7-97cf-a1ef08179e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:30.388856 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36375590-3e87-47d7-97cf-a1ef08179e9e admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:30.389141 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36375590-3e87-47d7-97cf-a1ef08179e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:30.389421 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36375590-3e87-47d7-97cf-a1ef08179e9e admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:30.390159 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36375590-3e87-47d7-97cf-a1ef08179e9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:30.392100 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36375590-3e87-47d7-97cf-a1ef08179e9e admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:30.396847 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 983/1934] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:30 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:30.396847 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1be2601-0eea-47c0-a170-c96bbbeb6949 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:30.407308 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1be2601-0eea-47c0-a170-c96bbbeb6949 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:30.412513 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b56abd-a6d8-4523-bd22-ca11aa5d4453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:30.413221 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b56abd-a6d8-4523-bd22-ca11aa5d4453 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:30.413569 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b56abd-a6d8-4523-bd22-ca11aa5d4453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:30.413875 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b56abd-a6d8-4523-bd22-ca11aa5d4453 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:30.414673 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b56abd-a6d8-4523-bd22-ca11aa5d4453 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:30.416381 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b56abd-a6d8-4523-bd22-ca11aa5d4453 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:30.424481 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 952/1935] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:41:30 2022] POST /identity/v3/projects => generated 371 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:30.425295 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36375590-3e87-47d7-97cf-a1ef08179e9e admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:30.428484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 984/1936] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:30.429441 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b56abd-a6d8-4523-bd22-ca11aa5d4453 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:30.438661 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:30.439755 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b56abd-a6d8-4523-bd22-ca11aa5d4453 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:30.494226 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36375590-3e87-47d7-97cf-a1ef08179e9e admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:30.499505 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83894ac4-b0b7-4f39-bb06-26b853123862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:30.500445 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83894ac4-b0b7-4f39-bb06-26b853123862 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:30.500733 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83894ac4-b0b7-4f39-bb06-26b853123862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:30.501013 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83894ac4-b0b7-4f39-bb06-26b853123862 admin admin] PATH_INFO: `/v3/projects/f5598cf5e0cc44b9918eb7d9c18db440/users/e35a6daa947442b78a30f49267f8cc16/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:30.502827 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2e5e41-11b7-4f53-834c-8064ea83a90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:30.503502 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e5e41-11b7-4f53-834c-8064ea83a90c admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:30.503840 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e5e41-11b7-4f53-834c-8064ea83a90c admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:30.505452 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e5e41-11b7-4f53-834c-8064ea83a90c admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:30.505452 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2e5e41-11b7-4f53-834c-8064ea83a90c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:30.507373 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2e5e41-11b7-4f53-834c-8064ea83a90c admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:30.521638 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:30.533082 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83894ac4-b0b7-4f39-bb06-26b853123862 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5598cf5e0cc44b9918eb7d9c18db440, user_id=e35a6daa947442b78a30f49267f8cc16, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:30.536476 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83894ac4-b0b7-4f39-bb06-26b853123862 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:30.553570 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 985/1937] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:41:30 2022] POST /identity/v3/users => generated 286 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:30.561615 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 953/1938] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:30 2022] PUT /identity/v3/projects/f5598cf5e0cc44b9918eb7d9c18db440/users/e35a6daa947442b78a30f49267f8cc16/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:30.562080 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2e5e41-11b7-4f53-834c-8064ea83a90c admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:30.568481 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83894ac4-b0b7-4f39-bb06-26b853123862 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:30.592056 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2e5e41-11b7-4f53-834c-8064ea83a90c admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:30.656557 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83894ac4-b0b7-4f39-bb06-26b853123862 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:30.662775 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d713d3-d4c4-4791-8ece-f2863ebc10e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:30.663455 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d713d3-d4c4-4791-8ece-f2863ebc10e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:30.663741 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d713d3-d4c4-4791-8ece-f2863ebc10e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:30.664022 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d713d3-d4c4-4791-8ece-f2863ebc10e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:30.664760 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d713d3-d4c4-4791-8ece-f2863ebc10e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:30.667169 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d713d3-d4c4-4791-8ece-f2863ebc10e8 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:30.681080 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d652bcd-6a43-4370-8398-1dd1f8246a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:30.681467 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 954/1939] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:30 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:30.682425 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d713d3-d4c4-4791-8ece-f2863ebc10e8 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:30.682622 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d652bcd-6a43-4370-8398-1dd1f8246a4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:30.682949 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d652bcd-6a43-4370-8398-1dd1f8246a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:30.683386 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d652bcd-6a43-4370-8398-1dd1f8246a4d admin admin] PATH_INFO: `/v3/projects/7f4018df46b041a2840da0a640e5c08e/users/16c03169f25844a2b1ef782b90aacf80/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:30.685975 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d652bcd-6a43-4370-8398-1dd1f8246a4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f4018df46b041a2840da0a640e5c08e, user_id=16c03169f25844a2b1ef782b90aacf80, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:30.688680 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d652bcd-6a43-4370-8398-1dd1f8246a4d admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:30.697785 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d713d3-d4c4-4791-8ece-f2863ebc10e8 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:30.705521 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 986/1940] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:30 2022] PUT /identity/v3/projects/7f4018df46b041a2840da0a640e5c08e/users/16c03169f25844a2b1ef782b90aacf80/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:30.707000 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d652bcd-6a43-4370-8398-1dd1f8246a4d admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:30.792994 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065f17b5-37d4-4e29-b290-86af62c6e3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:30.794133 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065f17b5-37d4-4e29-b290-86af62c6e3d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:30.795006 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065f17b5-37d4-4e29-b290-86af62c6e3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:30.795006 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065f17b5-37d4-4e29-b290-86af62c6e3d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:30.795434 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065f17b5-37d4-4e29-b290-86af62c6e3d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:30.797162 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065f17b5-37d4-4e29-b290-86af62c6e3d4 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:30.814181 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 955/1941] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:30 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:30.815790 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2995ca-d0c1-4fb4-ba06-79e643757342 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:30.816136 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2995ca-d0c1-4fb4-ba06-79e643757342 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:30.817054 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2995ca-d0c1-4fb4-ba06-79e643757342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:30.833838 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d652bcd-6a43-4370-8398-1dd1f8246a4d admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:30.840189 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81d822a-035d-4ed6-ad30-495730e09bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:30.841188 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d822a-035d-4ed6-ad30-495730e09bc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:30.841673 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d822a-035d-4ed6-ad30-495730e09bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:30.842439 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d822a-035d-4ed6-ad30-495730e09bc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:30.843636 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81d822a-035d-4ed6-ad30-495730e09bc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:30.845654 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81d822a-035d-4ed6-ad30-495730e09bc5 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:30.849352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-bb2995ca-d0c1-4fb4-ba06-79e643757342 None None] MFA Rules not processed for user `16c03169f25844a2b1ef782b90aacf80`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:30.850636 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-bb2995ca-d0c1-4fb4-ba06-79e643757342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:30.861223 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 987/1942] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:30 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:30.862415 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81d822a-035d-4ed6-ad30-495730e09bc5 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:30.871587 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:30.879859 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81d822a-035d-4ed6-ad30-495730e09bc5 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:30.890939 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2dea16-c0c5-4d05-b8a8-6d16c83cac34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:30.893508 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2dea16-c0c5-4d05-b8a8-6d16c83cac34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:30.894164 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2dea16-c0c5-4d05-b8a8-6d16c83cac34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:30.894629 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2dea16-c0c5-4d05-b8a8-6d16c83cac34 admin admin] PATH_INFO: `/v3/projects/f5598cf5e0cc44b9918eb7d9c18db440/users/e35a6daa947442b78a30f49267f8cc16/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:30.900743 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2dea16-c0c5-4d05-b8a8-6d16c83cac34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5598cf5e0cc44b9918eb7d9c18db440, user_id=e35a6daa947442b78a30f49267f8cc16, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:30.906818 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2dea16-c0c5-4d05-b8a8-6d16c83cac34 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:30.929773 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 988/1943] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:30 2022] PUT /identity/v3/projects/f5598cf5e0cc44b9918eb7d9c18db440/users/e35a6daa947442b78a30f49267f8cc16/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:30.930741 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2dea16-c0c5-4d05-b8a8-6d16c83cac34 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:30.974394 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 956/1944] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:30 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:30.977800 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0eedbe-5ccd-4b2c-abe2-807d37c2977b None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:30.978088 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0eedbe-5ccd-4b2c-abe2-807d37c2977b None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:30.978658 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0eedbe-5ccd-4b2c-abe2-807d37c2977b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:31.014884 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-ec0eedbe-5ccd-4b2c-abe2-807d37c2977b None None] MFA Rules not processed for user `e35a6daa947442b78a30f49267f8cc16`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:31.016251 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-ec0eedbe-5ccd-4b2c-abe2-807d37c2977b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:31.032687 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2dea16-c0c5-4d05-b8a8-6d16c83cac34 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:31.043882 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ab66e-33d8-4e5f-beba-a39fb86167d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:31.047835 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ab66e-33d8-4e5f-beba-a39fb86167d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:31.047835 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ab66e-33d8-4e5f-beba-a39fb86167d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:31.047835 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ab66e-33d8-4e5f-beba-a39fb86167d3 admin admin] PATH_INFO: `/v3/projects/91137dbd48fe4e6d986c350c8435bbab/users/740d843e5abb439b954996aaf841ed63/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:31.068217 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5ab66e-33d8-4e5f-beba-a39fb86167d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91137dbd48fe4e6d986c350c8435bbab, user_id=740d843e5abb439b954996aaf841ed63, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:31.072191 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5ab66e-33d8-4e5f-beba-a39fb86167d3 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:31.076388 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:31.109231 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 989/1945] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:30 2022] PUT /identity/v3/projects/91137dbd48fe4e6d986c350c8435bbab/users/740d843e5abb439b954996aaf841ed63/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:31.110764 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ab66e-33d8-4e5f-beba-a39fb86167d3 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:31.113876 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-7d5ab66e-33d8-4e5f-beba-a39fb86167d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:31.123441 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 957/1946] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:30 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:31.129034 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c36f99-e45b-482d-b6eb-db2bfae3b266 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:31.129812 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c36f99-e45b-482d-b6eb-db2bfae3b266 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:31.130238 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c36f99-e45b-482d-b6eb-db2bfae3b266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:31.153039 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-b1c36f99-e45b-482d-b6eb-db2bfae3b266 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:31.154417 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-b1c36f99-e45b-482d-b6eb-db2bfae3b266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:31.194898 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ab66e-33d8-4e5f-beba-a39fb86167d3 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:31.199368 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11b617b-b337-4fa7-9f26-f6dcec0e7887 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:31.200231 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11b617b-b337-4fa7-9f26-f6dcec0e7887 service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:31.200454 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11b617b-b337-4fa7-9f26-f6dcec0e7887 service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:31.200778 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11b617b-b337-4fa7-9f26-f6dcec0e7887 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:31.240278 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 958/1947] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:31.241168 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c36f99-e45b-482d-b6eb-db2bfae3b266 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:31.245124 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-b1c36f99-e45b-482d-b6eb-db2bfae3b266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:31.278642 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c36f99-e45b-482d-b6eb-db2bfae3b266 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:31.286118 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5434313d-02ca-49be-a318-c9092b61e963 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:31.286363 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5434313d-02ca-49be-a318-c9092b61e963 service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:31.286692 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5434313d-02ca-49be-a318-c9092b61e963 service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:31.287039 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5434313d-02ca-49be-a318-c9092b61e963 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:31.308378 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11b617b-b337-4fa7-9f26-f6dcec0e7887 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:31.311492 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11b617b-b337-4fa7-9f26-f6dcec0e7887 service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:31.337918 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:31.339657 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 990/1948] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:31 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:31.341163 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11b617b-b337-4fa7-9f26-f6dcec0e7887 service cinder] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:31.343479 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-c11b617b-b337-4fa7-9f26-f6dcec0e7887 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:31.470626 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5434313d-02ca-49be-a318-c9092b61e963 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:31.473540 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5434313d-02ca-49be-a318-c9092b61e963 service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:31.492480 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11b617b-b337-4fa7-9f26-f6dcec0e7887 service cinder] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:31.501213 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 959/1949] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:31 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:31.503169 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5434313d-02ca-49be-a318-c9092b61e963 service cinder] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:31.504695 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63f64fe-9658-4559-bc0d-1c3915b448bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:31.507562 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63f64fe-9658-4559-bc0d-1c3915b448bb service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:31.507721 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63f64fe-9658-4559-bc0d-1c3915b448bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:31.507998 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63f64fe-9658-4559-bc0d-1c3915b448bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:31.522628 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5434313d-02ca-49be-a318-c9092b61e963 service cinder] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:31.527883 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63f64fe-9658-4559-bc0d-1c3915b448bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:31.529344 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63f64fe-9658-4559-bc0d-1c3915b448bb service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:31.536422 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfef6bbe-e9d8-443b-ab2e-829343e81736 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:31.540580 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef6bbe-e9d8-443b-ab2e-829343e81736 service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:31.541836 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef6bbe-e9d8-443b-ab2e-829343e81736 service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:31.542426 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef6bbe-e9d8-443b-ab2e-829343e81736 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:31.544455 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 991/1950] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:31.547225 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-bfef6bbe-e9d8-443b-ab2e-829343e81736 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:31.700571 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfef6bbe-e9d8-443b-ab2e-829343e81736 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:31.703938 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfef6bbe-e9d8-443b-ab2e-829343e81736 service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:31.720974 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 960/1951] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:31 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:32.279065 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63f64fe-9658-4559-bc0d-1c3915b448bb service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:32.296851 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63f64fe-9658-4559-bc0d-1c3915b448bb service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:32.305243 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7066ae69-c937-41b1-818d-efbc965594e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:32.306637 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066ae69-c937-41b1-818d-efbc965594e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:32.307135 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066ae69-c937-41b1-818d-efbc965594e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:32.308076 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066ae69-c937-41b1-818d-efbc965594e1 admin admin] PATH_INFO: `/v3/users/b099108612094ff5b0d20e63b4259fa8` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:32.311272 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7066ae69-c937-41b1-818d-efbc965594e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b099108612094ff5b0d20e63b4259fa8)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:32.315347 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7066ae69-c937-41b1-818d-efbc965594e1 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:32.393122 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:INSERT=1,keystone:SELECT=93,keystone:DELETE=6 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:32.411920 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7066ae69-c937-41b1-818d-efbc965594e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b099108612094ff5b0d20e63b4259fa8'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:32.436764 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7066ae69-c937-41b1-818d-efbc965594e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b099108612094ff5b0d20e63b4259fa8'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:32.449156 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7066ae69-c937-41b1-818d-efbc965594e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b099108612094ff5b0d20e63b4259fa8'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:32.449905 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7066ae69-c937-41b1-818d-efbc965594e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b099108612094ff5b0d20e63b4259fa8'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:32.450756 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7066ae69-c937-41b1-818d-efbc965594e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b099108612094ff5b0d20e63b4259fa8'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:32.474254 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 992/1952] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:41:32 2022] DELETE /identity/v3/users/b099108612094ff5b0d20e63b4259fa8 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:32.478691 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e48707-aa1d-42db-92a4-dbe370ef7878 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:32.479165 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e48707-aa1d-42db-92a4-dbe370ef7878 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:32.479281 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e48707-aa1d-42db-92a4-dbe370ef7878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:32.499141 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-76e48707-aa1d-42db-92a4-dbe370ef7878 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:32.500362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-76e48707-aa1d-42db-92a4-dbe370ef7878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:32.570703 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0638d41-b9a5-4fa4-a1b3-18442eb4cd2b None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:32.571780 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0638d41-b9a5-4fa4-a1b3-18442eb4cd2b None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:32.572068 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0638d41-b9a5-4fa4-a1b3-18442eb4cd2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:32.583951 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 961/1953] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:32.594104 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e48707-aa1d-42db-92a4-dbe370ef7878 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:32.596965 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-76e48707-aa1d-42db-92a4-dbe370ef7878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:32.604925 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-b0638d41-b9a5-4fa4-a1b3-18442eb4cd2b None None] MFA Rules not processed for user `740d843e5abb439b954996aaf841ed63`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:32.606357 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b0638d41-b9a5-4fa4-a1b3-18442eb4cd2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:32.663098 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e48707-aa1d-42db-92a4-dbe370ef7878 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:32.667093 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3df2ae-41bf-4aff-9463-1e3ff12f8514 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:32.667742 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3df2ae-41bf-4aff-9463-1e3ff12f8514 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:32.668137 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3df2ae-41bf-4aff-9463-1e3ff12f8514 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:32.668298 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3df2ae-41bf-4aff-9463-1e3ff12f8514 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:32.679882 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3df2ae-41bf-4aff-9463-1e3ff12f8514 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:32.681678 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3df2ae-41bf-4aff-9463-1e3ff12f8514 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:32.690790 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 993/1954] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:41:32.695228 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 962/1955] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:41:32.702743 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0638d41-b9a5-4fa4-a1b3-18442eb4cd2b None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:32.704226 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b0638d41-b9a5-4fa4-a1b3-18442eb4cd2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:32.778112 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0638d41-b9a5-4fa4-a1b3-18442eb4cd2b None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:32.785665 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fafd70d-d4ad-480f-88cf-b3c488fd087f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:32.786373 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fafd70d-d4ad-480f-88cf-b3c488fd087f service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:32.786649 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fafd70d-d4ad-480f-88cf-b3c488fd087f service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:32.786932 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fafd70d-d4ad-480f-88cf-b3c488fd087f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:32.798124 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fafd70d-d4ad-480f-88cf-b3c488fd087f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:32.799686 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fafd70d-d4ad-480f-88cf-b3c488fd087f service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:32.812695 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 994/1956] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:32 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:33.295235 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3df2ae-41bf-4aff-9463-1e3ff12f8514 service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:33.297006 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-ef3df2ae-41bf-4aff-9463-1e3ff12f8514 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:33.317426 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3df2ae-41bf-4aff-9463-1e3ff12f8514 service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:33.325132 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e2568-2dbe-4ea8-8a93-f6ce0ed0b3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:33.326042 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e2568-2dbe-4ea8-8a93-f6ce0ed0b3b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:33.326351 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e2568-2dbe-4ea8-8a93-f6ce0ed0b3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:33.326703 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e2568-2dbe-4ea8-8a93-f6ce0ed0b3b9 admin admin] PATH_INFO: `/v3/projects/82c77153f8b14e028c543cf16a408c85` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:33.328161 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915e2568-2dbe-4ea8-8a93-f6ce0ed0b3b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82c77153f8b14e028c543cf16a408c85)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:33.332610 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915e2568-2dbe-4ea8-8a93-f6ce0ed0b3b9 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:33.344075 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:33.428736 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-915e2568-2dbe-4ea8-8a93-f6ce0ed0b3b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82c77153f8b14e028c543cf16a408c85', 'request_id': 'req-915e2568-2dbe-4ea8-8a93-f6ce0ed0b3b9'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:33.438371 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-915e2568-2dbe-4ea8-8a93-f6ce0ed0b3b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82c77153f8b14e028c543cf16a408c85', 'request_id': 'req-915e2568-2dbe-4ea8-8a93-f6ce0ed0b3b9'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:33.454277 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-915e2568-2dbe-4ea8-8a93-f6ce0ed0b3b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82c77153f8b14e028c543cf16a408c85', 'request_id': 'req-915e2568-2dbe-4ea8-8a93-f6ce0ed0b3b9'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:33.471343 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-915e2568-2dbe-4ea8-8a93-f6ce0ed0b3b9 admin admin] The token cache is being invalidate because project 82c77153f8b14e028c543cf16a408c85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:41:33.471711 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-915e2568-2dbe-4ea8-8a93-f6ce0ed0b3b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:33.473170 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-915e2568-2dbe-4ea8-8a93-f6ce0ed0b3b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:33.475470 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 963/1957] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:41:33 2022] DELETE /identity/v3/projects/82c77153f8b14e028c543cf16a408c85 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:33.479184 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fafd70d-d4ad-480f-88cf-b3c488fd087f service cinder] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:33.492887 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fafd70d-d4ad-480f-88cf-b3c488fd087f service cinder] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:33.564284 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:33.589282 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10c00c5-511b-4387-8cab-23a1d6176414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:33.590672 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10c00c5-511b-4387-8cab-23a1d6176414 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:33.590953 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10c00c5-511b-4387-8cab-23a1d6176414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:33.591222 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10c00c5-511b-4387-8cab-23a1d6176414 admin admin] PATH_INFO: `/v3/users/e35a6daa947442b78a30f49267f8cc16` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:33.594634 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10c00c5-511b-4387-8cab-23a1d6176414 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e35a6daa947442b78a30f49267f8cc16)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:33.600245 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10c00c5-511b-4387-8cab-23a1d6176414 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:33.676444 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d10c00c5-511b-4387-8cab-23a1d6176414 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e35a6daa947442b78a30f49267f8cc16'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:33.702210 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d10c00c5-511b-4387-8cab-23a1d6176414 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e35a6daa947442b78a30f49267f8cc16'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:33.724981 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d10c00c5-511b-4387-8cab-23a1d6176414 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e35a6daa947442b78a30f49267f8cc16'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:33.727613 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d10c00c5-511b-4387-8cab-23a1d6176414 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e35a6daa947442b78a30f49267f8cc16'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:33.728322 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d10c00c5-511b-4387-8cab-23a1d6176414 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e35a6daa947442b78a30f49267f8cc16'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:33.748877 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 995/1958] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:41:33 2022] DELETE /identity/v3/users/e35a6daa947442b78a30f49267f8cc16 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:41:34.286208 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415dac21-4f3e-45fe-b10c-013ac0ffe29b None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:34.286780 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415dac21-4f3e-45fe-b10c-013ac0ffe29b None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:34.286780 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415dac21-4f3e-45fe-b10c-013ac0ffe29b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:34.314772 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-415dac21-4f3e-45fe-b10c-013ac0ffe29b None None] MFA Rules not processed for user `740d843e5abb439b954996aaf841ed63`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:34.316113 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-415dac21-4f3e-45fe-b10c-013ac0ffe29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:34.389587 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 964/1959] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:34 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:34.441709 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10c00c5-511b-4387-8cab-23a1d6176414 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:34.448886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-d10c00c5-511b-4387-8cab-23a1d6176414 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:34.554727 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10c00c5-511b-4387-8cab-23a1d6176414 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:34.561344 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac77276-1530-4abe-b1e1-27988c59a674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:34.562760 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac77276-1530-4abe-b1e1-27988c59a674 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:34.563053 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac77276-1530-4abe-b1e1-27988c59a674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:34.563341 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac77276-1530-4abe-b1e1-27988c59a674 admin admin] PATH_INFO: `/v3/projects/f5598cf5e0cc44b9918eb7d9c18db440` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:34.564714 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac77276-1530-4abe-b1e1-27988c59a674 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5598cf5e0cc44b9918eb7d9c18db440)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:34.566913 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac77276-1530-4abe-b1e1-27988c59a674 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:34.624439 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-6ac77276-1530-4abe-b1e1-27988c59a674 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5598cf5e0cc44b9918eb7d9c18db440', 'request_id': 'req-6ac77276-1530-4abe-b1e1-27988c59a674'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:34.632207 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-6ac77276-1530-4abe-b1e1-27988c59a674 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5598cf5e0cc44b9918eb7d9c18db440', 'request_id': 'req-6ac77276-1530-4abe-b1e1-27988c59a674'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:34.638408 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-6ac77276-1530-4abe-b1e1-27988c59a674 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5598cf5e0cc44b9918eb7d9c18db440', 'request_id': 'req-6ac77276-1530-4abe-b1e1-27988c59a674'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:34.654320 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-6ac77276-1530-4abe-b1e1-27988c59a674 admin admin] The token cache is being invalidate because project f5598cf5e0cc44b9918eb7d9c18db440 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:41:34.654757 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-6ac77276-1530-4abe-b1e1-27988c59a674 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:34.655640 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-6ac77276-1530-4abe-b1e1-27988c59a674 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:34.661148 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 996/1960] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:41:34 2022] DELETE /identity/v3/projects/f5598cf5e0cc44b9918eb7d9c18db440 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:34.661148 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:34.662946 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b987d-6b67-4cb7-8dc6-3c5c92d6a08f None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:34.663328 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b987d-6b67-4cb7-8dc6-3c5c92d6a08f None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:34.663687 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b987d-6b67-4cb7-8dc6-3c5c92d6a08f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:34.685770 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-ea7b987d-6b67-4cb7-8dc6-3c5c92d6a08f None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:34.693144 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-ea7b987d-6b67-4cb7-8dc6-3c5c92d6a08f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:34.787650 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 965/1961] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:34.795519 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a195542-dc9d-45c3-99b3-6584ae144986 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:34.795817 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a195542-dc9d-45c3-99b3-6584ae144986 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:34.796095 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a195542-dc9d-45c3-99b3-6584ae144986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:34.821715 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-6a195542-dc9d-45c3-99b3-6584ae144986 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:34.823236 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-6a195542-dc9d-45c3-99b3-6584ae144986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:34.831952 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 997/1962] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:34.836239 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7b987d-6b67-4cb7-8dc6-3c5c92d6a08f None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:34.848205 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7b987d-6b67-4cb7-8dc6-3c5c92d6a08f None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:34.855608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e803d5-a025-4595-97c2-80aa25157d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:34.856469 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e803d5-a025-4595-97c2-80aa25157d4c admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:34.857067 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e803d5-a025-4595-97c2-80aa25157d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:34.857527 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e803d5-a025-4595-97c2-80aa25157d4c admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:34.858951 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e803d5-a025-4595-97c2-80aa25157d4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:41:34.859429 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e803d5-a025-4595-97c2-80aa25157d4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:34.867990 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e803d5-a025-4595-97c2-80aa25157d4c admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:34.899699 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 966/1963] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:41:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:41:34.900508 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a195542-dc9d-45c3-99b3-6584ae144986 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:34.912447 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a195542-dc9d-45c3-99b3-6584ae144986 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:34.919394 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699cbc88-a2d0-46c8-b9c5-0348365738f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:34.919394 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699cbc88-a2d0-46c8-b9c5-0348365738f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:34.919394 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699cbc88-a2d0-46c8-b9c5-0348365738f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:34.919394 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699cbc88-a2d0-46c8-b9c5-0348365738f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:34.919772 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699cbc88-a2d0-46c8-b9c5-0348365738f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:34.921920 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699cbc88-a2d0-46c8-b9c5-0348365738f2 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:34.961972 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 998/1964] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:41:34 2022] POST /identity/v3/projects => generated 381 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:34.970924 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e803d5-a025-4595-97c2-80aa25157d4c admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:34.989748 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e803d5-a025-4595-97c2-80aa25157d4c admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:35.040992 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:35.133288 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e2670-8ba0-43f1-ae56-44ef6b1a71a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:35.134258 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e2670-8ba0-43f1-ae56-44ef6b1a71a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:35.134823 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e2670-8ba0-43f1-ae56-44ef6b1a71a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:35.135286 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e2670-8ba0-43f1-ae56-44ef6b1a71a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:35.136258 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7e2670-8ba0-43f1-ae56-44ef6b1a71a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:35.143606 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7e2670-8ba0-43f1-ae56-44ef6b1a71a2 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:35.172937 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 967/1965] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:41:34 2022] POST /identity/v3/users => generated 291 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:35.176983 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699cbc88-a2d0-46c8-b9c5-0348365738f2 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:35.193995 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699cbc88-a2d0-46c8-b9c5-0348365738f2 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:35.208541 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60d0a81-a382-4e78-bdd6-510d97c327f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:35.209413 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d0a81-a382-4e78-bdd6-510d97c327f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:35.209888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d0a81-a382-4e78-bdd6-510d97c327f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:35.210316 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d0a81-a382-4e78-bdd6-510d97c327f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:35.211245 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60d0a81-a382-4e78-bdd6-510d97c327f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:35.212903 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60d0a81-a382-4e78-bdd6-510d97c327f6 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:35.237332 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 999/1966] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:35 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:35.249484 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e2670-8ba0-43f1-ae56-44ef6b1a71a2 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:35.261230 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e2670-8ba0-43f1-ae56-44ef6b1a71a2 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:35.266801 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8199c9-c477-47bf-be85-f688dec1b271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:35.267482 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8199c9-c477-47bf-be85-f688dec1b271 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:35.267768 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8199c9-c477-47bf-be85-f688dec1b271 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:35.268060 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8199c9-c477-47bf-be85-f688dec1b271 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:35.268943 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8199c9-c477-47bf-be85-f688dec1b271 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:35.270781 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8199c9-c477-47bf-be85-f688dec1b271 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:35.283031 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 968/1967] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:41:35.288337 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60d0a81-a382-4e78-bdd6-510d97c327f6 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:35.300191 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60d0a81-a382-4e78-bdd6-510d97c327f6 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:35.314288 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8905ae-ac0c-4350-bf84-cc8f34dea75b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:35.315401 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8905ae-ac0c-4350-bf84-cc8f34dea75b admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:35.315804 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8905ae-ac0c-4350-bf84-cc8f34dea75b admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:35.315988 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8905ae-ac0c-4350-bf84-cc8f34dea75b admin admin] PATH_INFO: `/v3/projects/a8bd2b76f8ac4b53ab8bb8c6099037e1/users/a25596e6a02b49e7b357e9b981f3ccf7/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:35.373984 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8905ae-ac0c-4350-bf84-cc8f34dea75b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8bd2b76f8ac4b53ab8bb8c6099037e1, user_id=a25596e6a02b49e7b357e9b981f3ccf7, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:35.373984 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8905ae-ac0c-4350-bf84-cc8f34dea75b admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:35.405118 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1000/1968] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:35 2022] PUT /identity/v3/projects/a8bd2b76f8ac4b53ab8bb8c6099037e1/users/a25596e6a02b49e7b357e9b981f3ccf7/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:35.418798 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a340262-4c2d-4160-b2c0-557917cddf4e None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:35.419385 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a340262-4c2d-4160-b2c0-557917cddf4e None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:35.420384 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a340262-4c2d-4160-b2c0-557917cddf4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:35.478588 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-7a340262-4c2d-4160-b2c0-557917cddf4e None None] MFA Rules not processed for user `a25596e6a02b49e7b357e9b981f3ccf7`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:35.480169 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-7a340262-4c2d-4160-b2c0-557917cddf4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:35.654483 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 969/1969] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:35.659747 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8905ae-ac0c-4350-bf84-cc8f34dea75b admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:35.776004 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8905ae-ac0c-4350-bf84-cc8f34dea75b admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:35.783354 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbce716a-ccc9-4489-adf0-863fbebf7754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:35.784550 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce716a-ccc9-4489-adf0-863fbebf7754 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:35.786146 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce716a-ccc9-4489-adf0-863fbebf7754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:35.787043 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce716a-ccc9-4489-adf0-863fbebf7754 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:35.787993 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbce716a-ccc9-4489-adf0-863fbebf7754 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:35.790451 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbce716a-ccc9-4489-adf0-863fbebf7754 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:35.830775 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1001/1970] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:41:35 2022] POST /identity/v3/projects => generated 383 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:35.839699 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a340262-4c2d-4160-b2c0-557917cddf4e None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:35.853981 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a340262-4c2d-4160-b2c0-557917cddf4e None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:35.930619 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838ea3e5-bd72-42b8-b6e4-82a1fcdbee23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:35.931362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ea3e5-bd72-42b8-b6e4-82a1fcdbee23 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:35.931654 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ea3e5-bd72-42b8-b6e4-82a1fcdbee23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:35.931941 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ea3e5-bd72-42b8-b6e4-82a1fcdbee23 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:35.932620 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838ea3e5-bd72-42b8-b6e4-82a1fcdbee23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:35.934770 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838ea3e5-bd72-42b8-b6e4-82a1fcdbee23 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:35.948303 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbce716a-ccc9-4489-adf0-863fbebf7754 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:35.950237 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-fbce716a-ccc9-4489-adf0-863fbebf7754 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:35.973450 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbce716a-ccc9-4489-adf0-863fbebf7754 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:35.975734 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 970/1971] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:41:35 2022] POST /identity/v3/users => generated 298 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:35.982856 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838ea3e5-bd72-42b8-b6e4-82a1fcdbee23 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:35.984418 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403c0a23-fa96-4b92-877f-f1c117b9436a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:35.985585 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c0a23-fa96-4b92-877f-f1c117b9436a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:35.986088 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c0a23-fa96-4b92-877f-f1c117b9436a admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:35.986497 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c0a23-fa96-4b92-877f-f1c117b9436a admin admin] PATH_INFO: `/v3/users/302f4ed3726d489b9dcd19969fc23c69` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:35.989007 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403c0a23-fa96-4b92-877f-f1c117b9436a admin admin] RBAC: Authorizing `identity:delete_user(user_id=302f4ed3726d489b9dcd19969fc23c69)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:35.992501 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403c0a23-fa96-4b92-877f-f1c117b9436a admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:36.001450 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838ea3e5-bd72-42b8-b6e4-82a1fcdbee23 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:36.007875 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ea4d4e-f6e0-4a43-a83d-e226d5e41693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:36.008558 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea4d4e-f6e0-4a43-a83d-e226d5e41693 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:36.008843 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea4d4e-f6e0-4a43-a83d-e226d5e41693 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:36.009160 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea4d4e-f6e0-4a43-a83d-e226d5e41693 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:36.009919 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ea4d4e-f6e0-4a43-a83d-e226d5e41693 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:36.011606 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ea4d4e-f6e0-4a43-a83d-e226d5e41693 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:36.027590 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 971/1972] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:35 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:36.028872 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ea4d4e-f6e0-4a43-a83d-e226d5e41693 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:36.043134 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ea4d4e-f6e0-4a43-a83d-e226d5e41693 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:36.050306 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afcfbd0-d8e8-4832-b0c8-cae394ff8e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:36.051381 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afcfbd0-d8e8-4832-b0c8-cae394ff8e00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:36.051802 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afcfbd0-d8e8-4832-b0c8-cae394ff8e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:36.052075 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afcfbd0-d8e8-4832-b0c8-cae394ff8e00 admin admin] PATH_INFO: `/v3/projects/b1ee98a746bd4f3ebd56e3bdbd73e508/users/ebc015436ddb42afa78f5a5b231faf31/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:36.066408 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:36.070331 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-403c0a23-fa96-4b92-877f-f1c117b9436a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '302f4ed3726d489b9dcd19969fc23c69'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:36.070990 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afcfbd0-d8e8-4832-b0c8-cae394ff8e00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1ee98a746bd4f3ebd56e3bdbd73e508, user_id=ebc015436ddb42afa78f5a5b231faf31, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:36.074733 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afcfbd0-d8e8-4832-b0c8-cae394ff8e00 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:36.085730 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-403c0a23-fa96-4b92-877f-f1c117b9436a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '302f4ed3726d489b9dcd19969fc23c69'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:36.092644 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 972/1973] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:36 2022] PUT /identity/v3/projects/b1ee98a746bd4f3ebd56e3bdbd73e508/users/ebc015436ddb42afa78f5a5b231faf31/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:41:36.095435 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-403c0a23-fa96-4b92-877f-f1c117b9436a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '302f4ed3726d489b9dcd19969fc23c69'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:36.096059 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-403c0a23-fa96-4b92-877f-f1c117b9436a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '302f4ed3726d489b9dcd19969fc23c69'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:36.096585 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afcfbd0-d8e8-4832-b0c8-cae394ff8e00 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:36.096761 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-403c0a23-fa96-4b92-877f-f1c117b9436a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '302f4ed3726d489b9dcd19969fc23c69'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:36.099304 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-2afcfbd0-d8e8-4832-b0c8-cae394ff8e00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:36.103437 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:36.119894 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1002/1974] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:41:35 2022] DELETE /identity/v3/users/302f4ed3726d489b9dcd19969fc23c69 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:36.126426 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a1251-9ea6-46be-b415-ff40c229891c None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:36.127066 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a1251-9ea6-46be-b415-ff40c229891c None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:36.127689 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a1251-9ea6-46be-b415-ff40c229891c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:36.164976 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-5b4a1251-9ea6-46be-b415-ff40c229891c None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:36.167013 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-5b4a1251-9ea6-46be-b415-ff40c229891c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:36.196896 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afcfbd0-d8e8-4832-b0c8-cae394ff8e00 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:36.245470 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1003/1975] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:36.258292 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a1251-9ea6-46be-b415-ff40c229891c None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:36.260138 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-5b4a1251-9ea6-46be-b415-ff40c229891c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:36.262797 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e1553-d6dd-438c-866f-77d5c9b5a5e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:36.263664 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e1553-d6dd-438c-866f-77d5c9b5a5e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:36.263959 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e1553-d6dd-438c-866f-77d5c9b5a5e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:36.264237 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e1553-d6dd-438c-866f-77d5c9b5a5e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:36.264986 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36e1553-d6dd-438c-866f-77d5c9b5a5e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:36.266700 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36e1553-d6dd-438c-866f-77d5c9b5a5e4 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:36.279297 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 973/1976] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:41:36.282937 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e1553-d6dd-438c-866f-77d5c9b5a5e4 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:36.295861 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e1553-d6dd-438c-866f-77d5c9b5a5e4 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:36.303567 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54e513a-f569-4058-b057-798ebff47d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:36.304473 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e513a-f569-4058-b057-798ebff47d18 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:36.304759 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e513a-f569-4058-b057-798ebff47d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:36.305038 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e513a-f569-4058-b057-798ebff47d18 admin admin] PATH_INFO: `/v3/projects/b1ee98a746bd4f3ebd56e3bdbd73e508/users/ebc015436ddb42afa78f5a5b231faf31/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:36.307381 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54e513a-f569-4058-b057-798ebff47d18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1ee98a746bd4f3ebd56e3bdbd73e508, user_id=ebc015436ddb42afa78f5a5b231faf31, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:36.310106 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54e513a-f569-4058-b057-798ebff47d18 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:36.321726 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 974/1977] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:36 2022] PUT /identity/v3/projects/b1ee98a746bd4f3ebd56e3bdbd73e508/users/ebc015436ddb42afa78f5a5b231faf31/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:41:36.332876 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b77bc-6097-465e-975b-606835ab18f6 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:36.333167 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b77bc-6097-465e-975b-606835ab18f6 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:36.333444 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b77bc-6097-465e-975b-606835ab18f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:36.342117 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a1251-9ea6-46be-b415-ff40c229891c None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:36.367576 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-4c0b77bc-6097-465e-975b-606835ab18f6 None None] MFA Rules not processed for user `ebc015436ddb42afa78f5a5b231faf31`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:36.373149 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-4c0b77bc-6097-465e-975b-606835ab18f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:36.400701 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c213df9e-fba6-47d1-a589-485203bdcb76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:36.401449 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c213df9e-fba6-47d1-a589-485203bdcb76 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:36.401854 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c213df9e-fba6-47d1-a589-485203bdcb76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:36.402203 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c213df9e-fba6-47d1-a589-485203bdcb76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:36.457854 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 975/1978] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:36 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:41:36.467946 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b77bc-6097-465e-975b-606835ab18f6 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:36.469380 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-4c0b77bc-6097-465e-975b-606835ab18f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:36.479286 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:36.479750 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c213df9e-fba6-47d1-a589-485203bdcb76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:36.481727 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c213df9e-fba6-47d1-a589-485203bdcb76 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:36.491693 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:36.500012 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1004/1979] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:41:36.546269 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b77bc-6097-465e-975b-606835ab18f6 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:36.551688 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d0c209-eddc-4bcc-b041-e253c5a171f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:36.552564 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d0c209-eddc-4bcc-b041-e253c5a171f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:36.553027 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d0c209-eddc-4bcc-b041-e253c5a171f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:36.554027 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d0c209-eddc-4bcc-b041-e253c5a171f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:36.581326 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d0c209-eddc-4bcc-b041-e253c5a171f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:36.583697 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d0c209-eddc-4bcc-b041-e253c5a171f4 service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:36.614749 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 976/1980] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:36 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:41:36.719652 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c213df9e-fba6-47d1-a589-485203bdcb76 service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:36.731011 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c213df9e-fba6-47d1-a589-485203bdcb76 service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:36.735893 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19e3679-35d0-49e2-8270-a37a5e00bb92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:36.736588 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e3679-35d0-49e2-8270-a37a5e00bb92 service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:36.736890 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e3679-35d0-49e2-8270-a37a5e00bb92 service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:36.737271 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e3679-35d0-49e2-8270-a37a5e00bb92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:36.739963 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-d19e3679-35d0-49e2-8270-a37a5e00bb92 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:36.821957 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19e3679-35d0-49e2-8270-a37a5e00bb92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:36.823770 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19e3679-35d0-49e2-8270-a37a5e00bb92 service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:36.828479 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d0c209-eddc-4bcc-b041-e253c5a171f4 service cinder] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:36.829969 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d2d0c209-eddc-4bcc-b041-e253c5a171f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:36.837627 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1005/1981] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:36 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:36.849810 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d0c209-eddc-4bcc-b041-e253c5a171f4 service cinder] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:36.857890 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cc188c-7031-4e42-8a23-81d3a38ff824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:36.858815 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc188c-7031-4e42-8a23-81d3a38ff824 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:36.859102 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc188c-7031-4e42-8a23-81d3a38ff824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:36.859380 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc188c-7031-4e42-8a23-81d3a38ff824 admin admin] PATH_INFO: `/v3/projects/d5f00d9c9e94486fbaf2297bab2fcad3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:36.860661 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cc188c-7031-4e42-8a23-81d3a38ff824 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5f00d9c9e94486fbaf2297bab2fcad3)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:36.862622 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cc188c-7031-4e42-8a23-81d3a38ff824 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:36.926598 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-b1cc188c-7031-4e42-8a23-81d3a38ff824 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5f00d9c9e94486fbaf2297bab2fcad3', 'request_id': 'req-b1cc188c-7031-4e42-8a23-81d3a38ff824'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:36.933827 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-b1cc188c-7031-4e42-8a23-81d3a38ff824 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5f00d9c9e94486fbaf2297bab2fcad3', 'request_id': 'req-b1cc188c-7031-4e42-8a23-81d3a38ff824'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:36.940180 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-b1cc188c-7031-4e42-8a23-81d3a38ff824 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5f00d9c9e94486fbaf2297bab2fcad3', 'request_id': 'req-b1cc188c-7031-4e42-8a23-81d3a38ff824'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:36.956026 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-b1cc188c-7031-4e42-8a23-81d3a38ff824 admin admin] The token cache is being invalidate because project d5f00d9c9e94486fbaf2297bab2fcad3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:41:36.956317 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-b1cc188c-7031-4e42-8a23-81d3a38ff824 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:36.957033 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-b1cc188c-7031-4e42-8a23-81d3a38ff824 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:36.958819 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 977/1982] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:41:36 2022] DELETE /identity/v3/projects/d5f00d9c9e94486fbaf2297bab2fcad3 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:41:36.962850 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19e3679-35d0-49e2-8270-a37a5e00bb92 service cinder] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:36.978392 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19e3679-35d0-49e2-8270-a37a5e00bb92 service cinder] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:37.058087 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3704ff-4724-41f6-bef0-0e28721cefc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:37.059049 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3704ff-4724-41f6-bef0-0e28721cefc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:37.059406 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3704ff-4724-41f6-bef0-0e28721cefc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:37.059761 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3704ff-4724-41f6-bef0-0e28721cefc0 admin admin] PATH_INFO: `/v3/users/16c03169f25844a2b1ef782b90aacf80` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:37.061277 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3704ff-4724-41f6-bef0-0e28721cefc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16c03169f25844a2b1ef782b90aacf80)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:37.063325 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3704ff-4724-41f6-bef0-0e28721cefc0 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:37.121334 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-ed3704ff-4724-41f6-bef0-0e28721cefc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16c03169f25844a2b1ef782b90aacf80'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:37.137020 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-ed3704ff-4724-41f6-bef0-0e28721cefc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16c03169f25844a2b1ef782b90aacf80'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:37.154987 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-ed3704ff-4724-41f6-bef0-0e28721cefc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16c03169f25844a2b1ef782b90aacf80'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:37.156385 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-ed3704ff-4724-41f6-bef0-0e28721cefc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16c03169f25844a2b1ef782b90aacf80'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:37.159354 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-ed3704ff-4724-41f6-bef0-0e28721cefc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16c03169f25844a2b1ef782b90aacf80'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:37.160612 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cc188c-7031-4e42-8a23-81d3a38ff824 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:37.162117 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-b1cc188c-7031-4e42-8a23-81d3a38ff824 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:37.190095 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cc188c-7031-4e42-8a23-81d3a38ff824 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:37.190297 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1006/1983] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:41:36 2022] DELETE /identity/v3/users/16c03169f25844a2b1ef782b90aacf80 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:37.279618 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d0298-c011-490d-8686-df61b3b014ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:37.281134 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d0298-c011-490d-8686-df61b3b014ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:37.281485 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d0298-c011-490d-8686-df61b3b014ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:37.281807 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d0298-c011-490d-8686-df61b3b014ed admin admin] PATH_INFO: `/v3/users/a25596e6a02b49e7b357e9b981f3ccf7` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:37.284640 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159d0298-c011-490d-8686-df61b3b014ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=a25596e6a02b49e7b357e9b981f3ccf7)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:37.288240 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159d0298-c011-490d-8686-df61b3b014ed admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:37.359270 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:37.372238 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-159d0298-c011-490d-8686-df61b3b014ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a25596e6a02b49e7b357e9b981f3ccf7'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:37.389710 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-159d0298-c011-490d-8686-df61b3b014ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a25596e6a02b49e7b357e9b981f3ccf7'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:37.390597 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-159d0298-c011-490d-8686-df61b3b014ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a25596e6a02b49e7b357e9b981f3ccf7'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:37.401750 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-159d0298-c011-490d-8686-df61b3b014ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a25596e6a02b49e7b357e9b981f3ccf7'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:37.402479 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-159d0298-c011-490d-8686-df61b3b014ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a25596e6a02b49e7b357e9b981f3ccf7'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:37.421377 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 978/1984] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:41:37 2022] DELETE /identity/v3/users/a25596e6a02b49e7b357e9b981f3ccf7 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:37.426301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd38eb6f-1d33-423a-88a5-79104e2ce680 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:37.426820 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd38eb6f-1d33-423a-88a5-79104e2ce680 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:37.427328 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd38eb6f-1d33-423a-88a5-79104e2ce680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:37.460776 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-cd38eb6f-1d33-423a-88a5-79104e2ce680 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:37.462824 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-cd38eb6f-1d33-423a-88a5-79104e2ce680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:37.479041 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:37.560670 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1007/1985] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:37.577988 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d0298-c011-490d-8686-df61b3b014ed admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:37.579434 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-159d0298-c011-490d-8686-df61b3b014ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:37.640029 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd38eb6f-1d33-423a-88a5-79104e2ce680 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:37.643724 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-cd38eb6f-1d33-423a-88a5-79104e2ce680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:37.664437 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd38eb6f-1d33-423a-88a5-79104e2ce680 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:37.669508 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a5e40-d3f2-4b8d-8d4a-26419efc76d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:37.670501 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a5e40-d3f2-4b8d-8d4a-26419efc76d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:37.670992 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a5e40-d3f2-4b8d-8d4a-26419efc76d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:37.671115 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a5e40-d3f2-4b8d-8d4a-26419efc76d8 admin admin] PATH_INFO: `/v3/projects/7f4018df46b041a2840da0a640e5c08e` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:37.672813 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571a5e40-d3f2-4b8d-8d4a-26419efc76d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f4018df46b041a2840da0a640e5c08e)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:37.675028 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571a5e40-d3f2-4b8d-8d4a-26419efc76d8 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:37.697102 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d0298-c011-490d-8686-df61b3b014ed admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:37.709155 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c495e943-af3e-4eb5-b999-09410281dca9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:37.709893 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c495e943-af3e-4eb5-b999-09410281dca9 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:37.710368 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c495e943-af3e-4eb5-b999-09410281dca9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:37.710661 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c495e943-af3e-4eb5-b999-09410281dca9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:37.723120 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c495e943-af3e-4eb5-b999-09410281dca9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:37.724768 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c495e943-af3e-4eb5-b999-09410281dca9 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:37.739918 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 979/1986] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:37.782985 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-571a5e40-d3f2-4b8d-8d4a-26419efc76d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f4018df46b041a2840da0a640e5c08e', 'request_id': 'req-571a5e40-d3f2-4b8d-8d4a-26419efc76d8'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:37.792510 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-571a5e40-d3f2-4b8d-8d4a-26419efc76d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f4018df46b041a2840da0a640e5c08e', 'request_id': 'req-571a5e40-d3f2-4b8d-8d4a-26419efc76d8'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:37.802131 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-571a5e40-d3f2-4b8d-8d4a-26419efc76d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f4018df46b041a2840da0a640e5c08e', 'request_id': 'req-571a5e40-d3f2-4b8d-8d4a-26419efc76d8'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:37.826851 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-571a5e40-d3f2-4b8d-8d4a-26419efc76d8 admin admin] The token cache is being invalidate because project 7f4018df46b041a2840da0a640e5c08e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:41:37.827679 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-571a5e40-d3f2-4b8d-8d4a-26419efc76d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:37.828730 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-571a5e40-d3f2-4b8d-8d4a-26419efc76d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:37.834410 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1008/1987] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:41:37 2022] DELETE /identity/v3/projects/7f4018df46b041a2840da0a640e5c08e => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:37.839239 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae7e15-8d2e-46c0-8bab-ee16216fc81a None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:37.839529 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae7e15-8d2e-46c0-8bab-ee16216fc81a None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:37.839910 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae7e15-8d2e-46c0-8bab-ee16216fc81a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:37.886769 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-84ae7e15-8d2e-46c0-8bab-ee16216fc81a None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:37.888282 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-84ae7e15-8d2e-46c0-8bab-ee16216fc81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:37.971010 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 980/1988] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:37.979330 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2b59d-1446-4cce-bec7-5d6754a7dfcc None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:37.979822 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2b59d-1446-4cce-bec7-5d6754a7dfcc None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:37.982409 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2b59d-1446-4cce-bec7-5d6754a7dfcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:38.012257 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-5be2b59d-1446-4cce-bec7-5d6754a7dfcc None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:38.013753 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-5be2b59d-1446-4cce-bec7-5d6754a7dfcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:38.023756 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1009/1989] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:38.028019 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae7e15-8d2e-46c0-8bab-ee16216fc81a None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:38.039312 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae7e15-8d2e-46c0-8bab-ee16216fc81a None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:38.044315 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f298560-268c-4817-adde-1c3fe658a33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:38.045114 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f298560-268c-4817-adde-1c3fe658a33c admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:38.045514 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f298560-268c-4817-adde-1c3fe658a33c admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:38.046053 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f298560-268c-4817-adde-1c3fe658a33c admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:38.046831 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f298560-268c-4817-adde-1c3fe658a33c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:41:38.048115 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f298560-268c-4817-adde-1c3fe658a33c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:38.048952 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f298560-268c-4817-adde-1c3fe658a33c admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:38.065411 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 981/1990] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:41:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:41:38.066907 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2b59d-1446-4cce-bec7-5d6754a7dfcc None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:38.076459 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2b59d-1446-4cce-bec7-5d6754a7dfcc None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:38.082646 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dbb60c-9202-4750-9096-1cc1a3face5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:38.083528 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dbb60c-9202-4750-9096-1cc1a3face5d admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:38.083884 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dbb60c-9202-4750-9096-1cc1a3face5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:38.084229 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dbb60c-9202-4750-9096-1cc1a3face5d admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:38.085006 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dbb60c-9202-4750-9096-1cc1a3face5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:38.087036 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dbb60c-9202-4750-9096-1cc1a3face5d admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:38.121610 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f298560-268c-4817-adde-1c3fe658a33c admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:38.123211 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1010/1991] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:41:38 2022] POST /identity/v3/projects => generated 369 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:38.124370 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dbb60c-9202-4750-9096-1cc1a3face5d admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:38.124858 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-3f298560-268c-4817-adde-1c3fe658a33c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:38.141588 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f298560-268c-4817-adde-1c3fe658a33c admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:38.190194 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:38.219879 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dbb60c-9202-4750-9096-1cc1a3face5d admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:38.224082 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454d44c-8305-43e2-aaf8-4d5ead2ac1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:38.225069 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454d44c-8305-43e2-aaf8-4d5ead2ac1c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:38.225484 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454d44c-8305-43e2-aaf8-4d5ead2ac1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:38.225590 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6bcaf7-94f3-44b7-bc86-d657e3a471ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:38.225964 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454d44c-8305-43e2-aaf8-4d5ead2ac1c1 admin admin] PATH_INFO: `/v3/projects/a8bd2b76f8ac4b53ab8bb8c6099037e1` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:38.226229 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6bcaf7-94f3-44b7-bc86-d657e3a471ba admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:38.226518 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6bcaf7-94f3-44b7-bc86-d657e3a471ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:38.226799 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6bcaf7-94f3-44b7-bc86-d657e3a471ba admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:38.227490 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6bcaf7-94f3-44b7-bc86-d657e3a471ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:38.227582 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2454d44c-8305-43e2-aaf8-4d5ead2ac1c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8bd2b76f8ac4b53ab8bb8c6099037e1)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:38.229421 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6bcaf7-94f3-44b7-bc86-d657e3a471ba admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:38.229753 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2454d44c-8305-43e2-aaf8-4d5ead2ac1c1 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:38.258663 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1011/1992] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:41:38 2022] POST /identity/v3/users => generated 285 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:38.259556 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6bcaf7-94f3-44b7-bc86-d657e3a471ba admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:38.265823 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:38.273326 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6bcaf7-94f3-44b7-bc86-d657e3a471ba admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:38.287635 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-2454d44c-8305-43e2-aaf8-4d5ead2ac1c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8bd2b76f8ac4b53ab8bb8c6099037e1', 'request_id': 'req-2454d44c-8305-43e2-aaf8-4d5ead2ac1c1'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:38.295041 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-2454d44c-8305-43e2-aaf8-4d5ead2ac1c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8bd2b76f8ac4b53ab8bb8c6099037e1', 'request_id': 'req-2454d44c-8305-43e2-aaf8-4d5ead2ac1c1'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:38.301115 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-2454d44c-8305-43e2-aaf8-4d5ead2ac1c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8bd2b76f8ac4b53ab8bb8c6099037e1', 'request_id': 'req-2454d44c-8305-43e2-aaf8-4d5ead2ac1c1'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:38.317323 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-2454d44c-8305-43e2-aaf8-4d5ead2ac1c1 admin admin] The token cache is being invalidate because project a8bd2b76f8ac4b53ab8bb8c6099037e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:41:38.317724 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-2454d44c-8305-43e2-aaf8-4d5ead2ac1c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:38.318432 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-2454d44c-8305-43e2-aaf8-4d5ead2ac1c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:38.320221 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 982/1993] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:41:38 2022] DELETE /identity/v3/projects/a8bd2b76f8ac4b53ab8bb8c6099037e1 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:41:38.326305 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454d44c-8305-43e2-aaf8-4d5ead2ac1c1 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:38.327841 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-2454d44c-8305-43e2-aaf8-4d5ead2ac1c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:38.354886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02797848-feee-4d39-9350-90d9428d4751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:38.355619 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02797848-feee-4d39-9350-90d9428d4751 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:38.355982 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02797848-feee-4d39-9350-90d9428d4751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:38.356623 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02797848-feee-4d39-9350-90d9428d4751 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:38.357770 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02797848-feee-4d39-9350-90d9428d4751 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:38.360831 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02797848-feee-4d39-9350-90d9428d4751 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:38.377765 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1012/1994] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:38 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:38.382197 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02797848-feee-4d39-9350-90d9428d4751 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:38.384031 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-02797848-feee-4d39-9350-90d9428d4751 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:38.402003 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02797848-feee-4d39-9350-90d9428d4751 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:38.411330 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b0a58-46b7-43c7-80d0-6edffd5ca209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:38.412376 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b0a58-46b7-43c7-80d0-6edffd5ca209 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:38.412728 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b0a58-46b7-43c7-80d0-6edffd5ca209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:38.413067 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b0a58-46b7-43c7-80d0-6edffd5ca209 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:38.414013 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9b0a58-46b7-43c7-80d0-6edffd5ca209 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:38.416222 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9b0a58-46b7-43c7-80d0-6edffd5ca209 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:38.437249 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454d44c-8305-43e2-aaf8-4d5ead2ac1c1 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:38.446049 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1013/1995] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:38 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:41:38.446255 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec0dbff-a561-4d3c-aae2-3f491597afd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:38.446973 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0dbff-a561-4d3c-aae2-3f491597afd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:38.447260 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0dbff-a561-4d3c-aae2-3f491597afd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:38.447395 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b0a58-46b7-43c7-80d0-6edffd5ca209 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:38.447528 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0dbff-a561-4d3c-aae2-3f491597afd2 admin admin] PATH_INFO: `/v3/users/ebc015436ddb42afa78f5a5b231faf31` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:38.449228 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec0dbff-a561-4d3c-aae2-3f491597afd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebc015436ddb42afa78f5a5b231faf31)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:38.453156 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec0dbff-a561-4d3c-aae2-3f491597afd2 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:38.461585 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b0a58-46b7-43c7-80d0-6edffd5ca209 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:38.467219 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0306bca-08b8-4fe5-9ceb-c30f9a1347b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:38.468327 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0306bca-08b8-4fe5-9ceb-c30f9a1347b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:38.468620 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0306bca-08b8-4fe5-9ceb-c30f9a1347b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:38.469379 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0306bca-08b8-4fe5-9ceb-c30f9a1347b9 admin admin] PATH_INFO: `/v3/projects/87582f0bf66948bea1f49ebb9d64dda3/users/c68c61c8f4954475b2e9cd9e0411755c/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:38.490644 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0306bca-08b8-4fe5-9ceb-c30f9a1347b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87582f0bf66948bea1f49ebb9d64dda3, user_id=c68c61c8f4954475b2e9cd9e0411755c, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:38.493357 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0306bca-08b8-4fe5-9ceb-c30f9a1347b9 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:38.512625 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1014/1996] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:38 2022] PUT /identity/v3/projects/87582f0bf66948bea1f49ebb9d64dda3/users/c68c61c8f4954475b2e9cd9e0411755c/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:38.525730 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde00137-7de3-4b3c-8e84-73f35dbc622a None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:38.526079 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde00137-7de3-4b3c-8e84-73f35dbc622a None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:38.527980 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde00137-7de3-4b3c-8e84-73f35dbc622a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:38.529368 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-fec0dbff-a561-4d3c-aae2-3f491597afd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebc015436ddb42afa78f5a5b231faf31'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:38.545890 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-fec0dbff-a561-4d3c-aae2-3f491597afd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebc015436ddb42afa78f5a5b231faf31'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:38.546895 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-fec0dbff-a561-4d3c-aae2-3f491597afd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebc015436ddb42afa78f5a5b231faf31'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:38.562674 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-bde00137-7de3-4b3c-8e84-73f35dbc622a None None] MFA Rules not processed for user `c68c61c8f4954475b2e9cd9e0411755c`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:38.563928 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-bde00137-7de3-4b3c-8e84-73f35dbc622a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:38.571268 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-fec0dbff-a561-4d3c-aae2-3f491597afd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebc015436ddb42afa78f5a5b231faf31'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:38.572276 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-fec0dbff-a561-4d3c-aae2-3f491597afd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebc015436ddb42afa78f5a5b231faf31'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:38.577816 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:38.599980 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 983/1997] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:41:38 2022] DELETE /identity/v3/users/ebc015436ddb42afa78f5a5b231faf31 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:38.677230 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1015/1998] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:38 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:38.690094 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec0dbff-a561-4d3c-aae2-3f491597afd2 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:38.692465 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-fec0dbff-a561-4d3c-aae2-3f491597afd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:38.763768 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec0dbff-a561-4d3c-aae2-3f491597afd2 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:38.768238 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba261fd9-3220-4266-a41a-c11bab5288f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:38.768979 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba261fd9-3220-4266-a41a-c11bab5288f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:38.769488 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba261fd9-3220-4266-a41a-c11bab5288f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:38.769966 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba261fd9-3220-4266-a41a-c11bab5288f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:38.866624 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba261fd9-3220-4266-a41a-c11bab5288f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:38.868131 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba261fd9-3220-4266-a41a-c11bab5288f7 service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:38.903050 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 984/1999] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:38 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:38.905166 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde00137-7de3-4b3c-8e84-73f35dbc622a None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:38.907229 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-bde00137-7de3-4b3c-8e84-73f35dbc622a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:38.977136 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:38.999781 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde00137-7de3-4b3c-8e84-73f35dbc622a None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:39.081234 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba261fd9-3220-4266-a41a-c11bab5288f7 service cinder] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:39.083805 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-ba261fd9-3220-4266-a41a-c11bab5288f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:39.108396 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c59c9a-0e16-4de0-9163-19890578f0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:39.109493 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c59c9a-0e16-4de0-9163-19890578f0eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:39.109813 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c59c9a-0e16-4de0-9163-19890578f0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:39.110091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c59c9a-0e16-4de0-9163-19890578f0eb admin admin] PATH_INFO: `/v3/users/c68c61c8f4954475b2e9cd9e0411755c` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:39.111828 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c59c9a-0e16-4de0-9163-19890578f0eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=c68c61c8f4954475b2e9cd9e0411755c)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:39.114746 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c59c9a-0e16-4de0-9163-19890578f0eb admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:39.196113 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-06c59c9a-0e16-4de0-9163-19890578f0eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c68c61c8f4954475b2e9cd9e0411755c'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:39.225434 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba261fd9-3220-4266-a41a-c11bab5288f7 service cinder] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:39.227890 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-06c59c9a-0e16-4de0-9163-19890578f0eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c68c61c8f4954475b2e9cd9e0411755c'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:39.230460 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e175e48c-04c7-49c0-a576-3a506bf434ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:39.231434 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175e48c-04c7-49c0-a576-3a506bf434ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:39.231639 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175e48c-04c7-49c0-a576-3a506bf434ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:39.231920 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175e48c-04c7-49c0-a576-3a506bf434ad admin admin] PATH_INFO: `/v3/projects/b1ee98a746bd4f3ebd56e3bdbd73e508` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:39.233170 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e175e48c-04c7-49c0-a576-3a506bf434ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1ee98a746bd4f3ebd56e3bdbd73e508)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:39.235167 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e175e48c-04c7-49c0-a576-3a506bf434ad admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:39.246601 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-06c59c9a-0e16-4de0-9163-19890578f0eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c68c61c8f4954475b2e9cd9e0411755c'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:39.247502 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-06c59c9a-0e16-4de0-9163-19890578f0eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c68c61c8f4954475b2e9cd9e0411755c'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:39.248583 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-06c59c9a-0e16-4de0-9163-19890578f0eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c68c61c8f4954475b2e9cd9e0411755c'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:39.265169 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1016/2000] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:41:38 2022] DELETE /identity/v3/users/c68c61c8f4954475b2e9cd9e0411755c => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:39.270871 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d2206c-8e5e-47b3-856a-8ef098505c89 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:39.271194 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d2206c-8e5e-47b3-856a-8ef098505c89 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:39.271517 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d2206c-8e5e-47b3-856a-8ef098505c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:39.294095 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-31d2206c-8e5e-47b3-856a-8ef098505c89 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:39.295691 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:INSERT=1,keystone:DELETE=5,keystone:SELECT=94 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:39.295819 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-31d2206c-8e5e-47b3-856a-8ef098505c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:39.297893 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-e175e48c-04c7-49c0-a576-3a506bf434ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1ee98a746bd4f3ebd56e3bdbd73e508', 'request_id': 'req-e175e48c-04c7-49c0-a576-3a506bf434ad'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:39.308045 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-e175e48c-04c7-49c0-a576-3a506bf434ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1ee98a746bd4f3ebd56e3bdbd73e508', 'request_id': 'req-e175e48c-04c7-49c0-a576-3a506bf434ad'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:39.314546 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-e175e48c-04c7-49c0-a576-3a506bf434ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1ee98a746bd4f3ebd56e3bdbd73e508', 'request_id': 'req-e175e48c-04c7-49c0-a576-3a506bf434ad'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:39.330802 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-e175e48c-04c7-49c0-a576-3a506bf434ad admin admin] The token cache is being invalidate because project b1ee98a746bd4f3ebd56e3bdbd73e508 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:41:39.331136 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-e175e48c-04c7-49c0-a576-3a506bf434ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:39.332143 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-e175e48c-04c7-49c0-a576-3a506bf434ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:39.334230 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 985/2001] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:41:39 2022] DELETE /identity/v3/projects/b1ee98a746bd4f3ebd56e3bdbd73e508 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:39.338953 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda1714-82ce-472a-b687-23a199703df4 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:39.339243 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda1714-82ce-472a-b687-23a199703df4 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:39.339524 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda1714-82ce-472a-b687-23a199703df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:39.359635 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-3eda1714-82ce-472a-b687-23a199703df4 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:39.360881 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-3eda1714-82ce-472a-b687-23a199703df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:39.386822 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1017/2002] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:41:39.397830 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d2206c-8e5e-47b3-856a-8ef098505c89 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:39.399273 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-31d2206c-8e5e-47b3-856a-8ef098505c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:39.427481 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:39.451067 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 986/2003] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:41:39.458965 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34448b5f-8afe-423c-87c2-90492586504c None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:39.459252 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34448b5f-8afe-423c-87c2-90492586504c None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:39.459524 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34448b5f-8afe-423c-87c2-90492586504c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:39.480415 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-34448b5f-8afe-423c-87c2-90492586504c None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:39.480955 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d2206c-8e5e-47b3-856a-8ef098505c89 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:39.482487 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-34448b5f-8afe-423c-87c2-90492586504c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:39.484882 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d338046d-78c6-4b48-9f98-0159b762527f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:39.486020 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d338046d-78c6-4b48-9f98-0159b762527f service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:39.486316 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d338046d-78c6-4b48-9f98-0159b762527f service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:39.487002 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d338046d-78c6-4b48-9f98-0159b762527f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:39.500271 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d338046d-78c6-4b48-9f98-0159b762527f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:39.501451 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 987/2004] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:41:39.501755 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d338046d-78c6-4b48-9f98-0159b762527f service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:39.502430 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34448b5f-8afe-423c-87c2-90492586504c None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:39.513524 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34448b5f-8afe-423c-87c2-90492586504c None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:39.514071 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1018/2005] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:39.519007 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b608a67b-2a07-4e5a-9848-60c0fba53c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:39.520416 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b608a67b-2a07-4e5a-9848-60c0fba53c6d admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:39.520699 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b608a67b-2a07-4e5a-9848-60c0fba53c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:39.520973 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b608a67b-2a07-4e5a-9848-60c0fba53c6d admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:39.521727 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b608a67b-2a07-4e5a-9848-60c0fba53c6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:41:39.522038 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b608a67b-2a07-4e5a-9848-60c0fba53c6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:39.523669 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b608a67b-2a07-4e5a-9848-60c0fba53c6d admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:39.537083 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 988/2006] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:41:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:41:39.542130 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d338046d-78c6-4b48-9f98-0159b762527f service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:39.554837 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d338046d-78c6-4b48-9f98-0159b762527f service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:39.559451 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d296c1-6870-4820-bb29-c7a4450cf2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:39.560235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d296c1-6870-4820-bb29-c7a4450cf2b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:39.560642 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d296c1-6870-4820-bb29-c7a4450cf2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:39.561590 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d296c1-6870-4820-bb29-c7a4450cf2b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:39.561761 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d296c1-6870-4820-bb29-c7a4450cf2b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:39.564086 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d296c1-6870-4820-bb29-c7a4450cf2b6 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:39.602133 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1019/2007] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:41:39 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:39.607264 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b608a67b-2a07-4e5a-9848-60c0fba53c6d admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:39.618832 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b608a67b-2a07-4e5a-9848-60c0fba53c6d admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:39.702032 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b236337e-7c76-443b-b983-16767a9a3d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:39.702829 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236337e-7c76-443b-b983-16767a9a3d81 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:39.703189 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236337e-7c76-443b-b983-16767a9a3d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:39.703987 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236337e-7c76-443b-b983-16767a9a3d81 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:39.704436 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b236337e-7c76-443b-b983-16767a9a3d81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:39.706496 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b236337e-7c76-443b-b983-16767a9a3d81 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:39.737046 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 989/2008] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:41:39 2022] POST /identity/v3/users => generated 293 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:41:39.741149 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d296c1-6870-4820-bb29-c7a4450cf2b6 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:39.751861 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d296c1-6870-4820-bb29-c7a4450cf2b6 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:39.757198 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f771fa-32a0-462a-baec-8439e78e3e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:39.757935 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f771fa-32a0-462a-baec-8439e78e3e84 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:39.758222 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f771fa-32a0-462a-baec-8439e78e3e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:39.758498 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f771fa-32a0-462a-baec-8439e78e3e84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:39.759231 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f771fa-32a0-462a-baec-8439e78e3e84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:39.760841 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f771fa-32a0-462a-baec-8439e78e3e84 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:39.775755 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1020/2009] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:39.777759 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b236337e-7c76-443b-b983-16767a9a3d81 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:39.788796 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b236337e-7c76-443b-b983-16767a9a3d81 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:39.794309 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d194d7a-bce5-4cd2-b524-210f9a1a267f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:39.795175 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d194d7a-bce5-4cd2-b524-210f9a1a267f admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:39.795521 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d194d7a-bce5-4cd2-b524-210f9a1a267f admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:39.795862 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d194d7a-bce5-4cd2-b524-210f9a1a267f admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:39.796649 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d194d7a-bce5-4cd2-b524-210f9a1a267f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:39.798375 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d194d7a-bce5-4cd2-b524-210f9a1a267f admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:39.809059 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 990/2010] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:39.813256 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f771fa-32a0-462a-baec-8439e78e3e84 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:39.823869 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f771fa-32a0-462a-baec-8439e78e3e84 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:39.829537 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b08265-203a-48bb-9d0b-9bfefc9327f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:39.830540 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b08265-203a-48bb-9d0b-9bfefc9327f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:39.830826 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b08265-203a-48bb-9d0b-9bfefc9327f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:39.831095 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b08265-203a-48bb-9d0b-9bfefc9327f6 admin admin] PATH_INFO: `/v3/projects/3bba6f91e5b34f2992f9aabf1a3725bf/users/200f52cbd2154662880ddf6292a2e9b4/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:39.848451 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b08265-203a-48bb-9d0b-9bfefc9327f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bba6f91e5b34f2992f9aabf1a3725bf, user_id=200f52cbd2154662880ddf6292a2e9b4, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:39.851135 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b08265-203a-48bb-9d0b-9bfefc9327f6 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:39.865886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1021/2011] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:39 2022] PUT /identity/v3/projects/3bba6f91e5b34f2992f9aabf1a3725bf/users/200f52cbd2154662880ddf6292a2e9b4/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:41:39.877398 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca643124-caf6-447e-9cf4-3e43c5df4850 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:39.877717 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca643124-caf6-447e-9cf4-3e43c5df4850 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:39.878001 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca643124-caf6-447e-9cf4-3e43c5df4850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:39.906632 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-ca643124-caf6-447e-9cf4-3e43c5df4850 None None] MFA Rules not processed for user `200f52cbd2154662880ddf6292a2e9b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:39.907873 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-ca643124-caf6-447e-9cf4-3e43c5df4850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:39.934094 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:39.979146 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 991/2012] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:39.983785 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b08265-203a-48bb-9d0b-9bfefc9327f6 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:40.078251 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b08265-203a-48bb-9d0b-9bfefc9327f6 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:40.085477 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc14e7a-0099-4f1d-823a-4290dda55642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:40.086600 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc14e7a-0099-4f1d-823a-4290dda55642 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:40.087157 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc14e7a-0099-4f1d-823a-4290dda55642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:40.087375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc14e7a-0099-4f1d-823a-4290dda55642 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:40.088133 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc14e7a-0099-4f1d-823a-4290dda55642 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:40.090074 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc14e7a-0099-4f1d-823a-4290dda55642 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:40.129905 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1022/2013] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:41:39 2022] POST /identity/v3/projects => generated 383 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:40.130791 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca643124-caf6-447e-9cf4-3e43c5df4850 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:40.141619 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca643124-caf6-447e-9cf4-3e43c5df4850 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:40.169492 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc14e7a-0099-4f1d-823a-4290dda55642 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:40.171322 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-bfc14e7a-0099-4f1d-823a-4290dda55642 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:40.239506 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f203c383-70a3-4223-9003-03ccdee1bbdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:40.239739 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203c383-70a3-4223-9003-03ccdee1bbdd admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:40.240100 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203c383-70a3-4223-9003-03ccdee1bbdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:40.240309 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203c383-70a3-4223-9003-03ccdee1bbdd admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:40.241023 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f203c383-70a3-4223-9003-03ccdee1bbdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:40.243580 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f203c383-70a3-4223-9003-03ccdee1bbdd admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:40.248829 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:40.273855 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 992/2014] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:41:40 2022] POST /identity/v3/users => generated 298 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:40.275909 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f203c383-70a3-4223-9003-03ccdee1bbdd admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:40.281415 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc14e7a-0099-4f1d-823a-4290dda55642 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:40.287589 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf2cd9a-d3a3-4181-91ae-4a3150a2475c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:40.287753 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f203c383-70a3-4223-9003-03ccdee1bbdd admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:40.288606 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2cd9a-d3a3-4181-91ae-4a3150a2475c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:40.288894 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2cd9a-d3a3-4181-91ae-4a3150a2475c admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:40.289175 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2cd9a-d3a3-4181-91ae-4a3150a2475c admin admin] PATH_INFO: `/v3/projects/87582f0bf66948bea1f49ebb9d64dda3` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:40.290377 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf2cd9a-d3a3-4181-91ae-4a3150a2475c admin admin] RBAC: Authorizing `identity:delete_project(project_id=87582f0bf66948bea1f49ebb9d64dda3)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:40.292301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf2cd9a-d3a3-4181-91ae-4a3150a2475c admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:40.292780 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5866701c-033c-4d2c-95a9-317cf001bd5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:40.293550 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5866701c-033c-4d2c-95a9-317cf001bd5b admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:40.294010 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5866701c-033c-4d2c-95a9-317cf001bd5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:40.294296 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5866701c-033c-4d2c-95a9-317cf001bd5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:40.295141 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5866701c-033c-4d2c-95a9-317cf001bd5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:40.297128 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5866701c-033c-4d2c-95a9-317cf001bd5b admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:40.313303 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 993/2015] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:41:40.314202 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5866701c-033c-4d2c-95a9-317cf001bd5b admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:40.325785 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5866701c-033c-4d2c-95a9-317cf001bd5b admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:40.331052 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6363c0-6264-4926-8db9-f981653b6a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:40.332943 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6363c0-6264-4926-8db9-f981653b6a86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:40.332943 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6363c0-6264-4926-8db9-f981653b6a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:40.333368 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6363c0-6264-4926-8db9-f981653b6a86 admin admin] PATH_INFO: `/v3/projects/cec7ff6a1abf415c9a2f50c73d0a44f3/users/93ed8954ab3149fba2a7749893e7f13b/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:40.355454 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-eaf2cd9a-d3a3-4181-91ae-4a3150a2475c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87582f0bf66948bea1f49ebb9d64dda3', 'request_id': 'req-eaf2cd9a-d3a3-4181-91ae-4a3150a2475c'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:40.361815 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6363c0-6264-4926-8db9-f981653b6a86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cec7ff6a1abf415c9a2f50c73d0a44f3, user_id=93ed8954ab3149fba2a7749893e7f13b, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:40.362001 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-eaf2cd9a-d3a3-4181-91ae-4a3150a2475c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87582f0bf66948bea1f49ebb9d64dda3', 'request_id': 'req-eaf2cd9a-d3a3-4181-91ae-4a3150a2475c'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:40.365850 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6363c0-6264-4926-8db9-f981653b6a86 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:40.375153 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-eaf2cd9a-d3a3-4181-91ae-4a3150a2475c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87582f0bf66948bea1f49ebb9d64dda3', 'request_id': 'req-eaf2cd9a-d3a3-4181-91ae-4a3150a2475c'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:40.388230 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 994/2016] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:40 2022] PUT /identity/v3/projects/cec7ff6a1abf415c9a2f50c73d0a44f3/users/93ed8954ab3149fba2a7749893e7f13b/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:40.391596 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-eaf2cd9a-d3a3-4181-91ae-4a3150a2475c admin admin] The token cache is being invalidate because project 87582f0bf66948bea1f49ebb9d64dda3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:41:40.391701 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-eaf2cd9a-d3a3-4181-91ae-4a3150a2475c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:40.392329 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-eaf2cd9a-d3a3-4181-91ae-4a3150a2475c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:40.392983 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6363c0-6264-4926-8db9-f981653b6a86 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:40.394099 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1023/2017] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:41:40 2022] DELETE /identity/v3/projects/87582f0bf66948bea1f49ebb9d64dda3 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:41:40.399292 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfdf653-ad18-4179-b9c3-df2a47350aec None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:40.399592 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfdf653-ad18-4179-b9c3-df2a47350aec None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:40.400066 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfdf653-ad18-4179-b9c3-df2a47350aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:40.402910 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-dd6363c0-6264-4926-8db9-f981653b6a86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:40.421185 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-ebfdf653-ad18-4179-b9c3-df2a47350aec None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:40.422595 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-ebfdf653-ad18-4179-b9c3-df2a47350aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:40.479555 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6363c0-6264-4926-8db9-f981653b6a86 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:40.485096 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911597bc-9664-4f4e-96da-040ca084ff68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:40.485790 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911597bc-9664-4f4e-96da-040ca084ff68 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:40.486075 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911597bc-9664-4f4e-96da-040ca084ff68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:40.486350 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911597bc-9664-4f4e-96da-040ca084ff68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:40.487111 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911597bc-9664-4f4e-96da-040ca084ff68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:40.489576 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911597bc-9664-4f4e-96da-040ca084ff68 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:40.493033 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1024/2018] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:41:40.500689 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511978b5-8d30-4ae7-9c70-ce74d789d163 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:40.500985 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511978b5-8d30-4ae7-9c70-ce74d789d163 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:40.501266 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511978b5-8d30-4ae7-9c70-ce74d789d163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:40.504545 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 995/2019] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:40 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:40.505603 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911597bc-9664-4f4e-96da-040ca084ff68 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:40.516412 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911597bc-9664-4f4e-96da-040ca084ff68 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:40.522525 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-511978b5-8d30-4ae7-9c70-ce74d789d163 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:40.522970 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af64ae30-9343-4e22-ba3a-479cbfba1006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:40.523744 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-511978b5-8d30-4ae7-9c70-ce74d789d163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:40.523860 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64ae30-9343-4e22-ba3a-479cbfba1006 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:40.524150 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64ae30-9343-4e22-ba3a-479cbfba1006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:40.524620 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64ae30-9343-4e22-ba3a-479cbfba1006 admin admin] PATH_INFO: `/v3/projects/cec7ff6a1abf415c9a2f50c73d0a44f3/users/93ed8954ab3149fba2a7749893e7f13b/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:40.526672 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af64ae30-9343-4e22-ba3a-479cbfba1006 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cec7ff6a1abf415c9a2f50c73d0a44f3, user_id=93ed8954ab3149fba2a7749893e7f13b, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:40.529343 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af64ae30-9343-4e22-ba3a-479cbfba1006 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:40.532440 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1025/2020] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:41:40.537870 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511978b5-8d30-4ae7-9c70-ce74d789d163 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:40.543321 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 996/2021] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:40 2022] PUT /identity/v3/projects/cec7ff6a1abf415c9a2f50c73d0a44f3/users/93ed8954ab3149fba2a7749893e7f13b/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:40.554098 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69812fac-78fb-4645-82ee-bf79133eeb94 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:40.554464 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69812fac-78fb-4645-82ee-bf79133eeb94 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:40.554747 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69812fac-78fb-4645-82ee-bf79133eeb94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:40.555979 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511978b5-8d30-4ae7-9c70-ce74d789d163 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:40.560852 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:40.586596 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-69812fac-78fb-4645-82ee-bf79133eeb94 None None] MFA Rules not processed for user `93ed8954ab3149fba2a7749893e7f13b`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:40.587861 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-69812fac-78fb-4645-82ee-bf79133eeb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:40.624058 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40277d3-bff6-4214-8452-3b73f2397080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:40.624740 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40277d3-bff6-4214-8452-3b73f2397080 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:40.625027 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40277d3-bff6-4214-8452-3b73f2397080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:40.625420 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40277d3-bff6-4214-8452-3b73f2397080 admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:40.626192 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40277d3-bff6-4214-8452-3b73f2397080 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:41:40.626613 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40277d3-bff6-4214-8452-3b73f2397080 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:40.630798 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40277d3-bff6-4214-8452-3b73f2397080 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:40.650551 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1026/2022] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:41:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:41:40.654959 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40277d3-bff6-4214-8452-3b73f2397080 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:40.667925 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40277d3-bff6-4214-8452-3b73f2397080 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:40.677275 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce4d22-09da-40db-ba64-e067ea0ced6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:40.678290 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce4d22-09da-40db-ba64-e067ea0ced6e admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:40.678741 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce4d22-09da-40db-ba64-e067ea0ced6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:40.679393 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce4d22-09da-40db-ba64-e067ea0ced6e admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:40.680396 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ce4d22-09da-40db-ba64-e067ea0ced6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:40.682521 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ce4d22-09da-40db-ba64-e067ea0ced6e admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:40.687895 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:40.698191 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 997/2023] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:40 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:41:40.708767 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69812fac-78fb-4645-82ee-bf79133eeb94 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:40.711316 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-69812fac-78fb-4645-82ee-bf79133eeb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:40.731190 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1027/2024] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:41:40 2022] POST /identity/v3/projects => generated 377 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:41:40.732774 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce4d22-09da-40db-ba64-e067ea0ced6e admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:40.803327 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69812fac-78fb-4645-82ee-bf79133eeb94 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:40.816450 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce4d22-09da-40db-ba64-e067ea0ced6e admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:40.822533 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75c170e-1fef-4a84-a2c8-a4fcba43547b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:40.823633 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75c170e-1fef-4a84-a2c8-a4fcba43547b admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:40.824100 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75c170e-1fef-4a84-a2c8-a4fcba43547b admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:40.824603 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75c170e-1fef-4a84-a2c8-a4fcba43547b admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:40.825568 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75c170e-1fef-4a84-a2c8-a4fcba43547b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:40.828812 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75c170e-1fef-4a84-a2c8-a4fcba43547b admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:40.857330 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1028/2025] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:41:40 2022] POST /identity/v3/users => generated 289 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:40.861750 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75c170e-1fef-4a84-a2c8-a4fcba43547b admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:40.869235 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fb8cf7-2817-45cc-a7be-e8637793e920 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:40.869235 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb8cf7-2817-45cc-a7be-e8637793e920 service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:40.874091 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb8cf7-2817-45cc-a7be-e8637793e920 service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:40.874388 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb8cf7-2817-45cc-a7be-e8637793e920 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:40.875056 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75c170e-1fef-4a84-a2c8-a4fcba43547b admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:40.879659 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d9fb8cf7-2817-45cc-a7be-e8637793e920 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:40.882788 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c90d55e-aba8-40c7-becf-aaf68b79b29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:40.883742 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90d55e-aba8-40c7-becf-aaf68b79b29e admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:40.884094 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90d55e-aba8-40c7-becf-aaf68b79b29e admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:40.884552 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90d55e-aba8-40c7-becf-aaf68b79b29e admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:40.885566 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c90d55e-aba8-40c7-becf-aaf68b79b29e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:40.887441 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c90d55e-aba8-40c7-becf-aaf68b79b29e admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:40.903895 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1029/2026] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:40 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:40.905158 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c90d55e-aba8-40c7-becf-aaf68b79b29e admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:40.916930 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c90d55e-aba8-40c7-becf-aaf68b79b29e admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:40.921800 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66600afe-4f35-4dc5-a745-2dda751fdf26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:40.922475 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66600afe-4f35-4dc5-a745-2dda751fdf26 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:40.922759 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66600afe-4f35-4dc5-a745-2dda751fdf26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:40.923083 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66600afe-4f35-4dc5-a745-2dda751fdf26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:40.923970 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66600afe-4f35-4dc5-a745-2dda751fdf26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:40.925937 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66600afe-4f35-4dc5-a745-2dda751fdf26 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:40.939067 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1030/2027] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:40.943553 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66600afe-4f35-4dc5-a745-2dda751fdf26 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:40.950825 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fb8cf7-2817-45cc-a7be-e8637793e920 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:40.952280 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fb8cf7-2817-45cc-a7be-e8637793e920 service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:40.957396 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:40.959161 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66600afe-4f35-4dc5-a745-2dda751fdf26 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:40.967007 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401fd258-cec7-4b2f-a1b9-bc6180aeed1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:40.967269 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 998/2028] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:41:40.968118 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401fd258-cec7-4b2f-a1b9-bc6180aeed1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:40.968555 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401fd258-cec7-4b2f-a1b9-bc6180aeed1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:40.968939 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401fd258-cec7-4b2f-a1b9-bc6180aeed1a admin admin] PATH_INFO: `/v3/projects/7d1284e4a0944fee92ffc37d2120306a/users/84a3046facfb40ec893eed6aae236a0c/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:40.992297 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401fd258-cec7-4b2f-a1b9-bc6180aeed1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d1284e4a0944fee92ffc37d2120306a, user_id=84a3046facfb40ec893eed6aae236a0c, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:40.996571 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401fd258-cec7-4b2f-a1b9-bc6180aeed1a admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:41.016440 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1031/2029] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:40 2022] PUT /identity/v3/projects/7d1284e4a0944fee92ffc37d2120306a/users/84a3046facfb40ec893eed6aae236a0c/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:41:41.026882 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a418abcb-ee0f-41a6-9d46-b33185fa44f5 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:41.027311 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a418abcb-ee0f-41a6-9d46-b33185fa44f5 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:41.027670 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a418abcb-ee0f-41a6-9d46-b33185fa44f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:41.061358 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-a418abcb-ee0f-41a6-9d46-b33185fa44f5 None None] MFA Rules not processed for user `84a3046facfb40ec893eed6aae236a0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:41.062715 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-a418abcb-ee0f-41a6-9d46-b33185fa44f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:41.140355 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 999/2030] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:41:41.153788 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401fd258-cec7-4b2f-a1b9-bc6180aeed1a admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:41.155765 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-401fd258-cec7-4b2f-a1b9-bc6180aeed1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:41.237271 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401fd258-cec7-4b2f-a1b9-bc6180aeed1a admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:41.242921 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206efc9b-af05-4da1-a108-73e2451f1a48 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd15c5f6aeb874191b73c8ee0410c6bc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:41.243963 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206efc9b-af05-4da1-a108-73e2451f1a48 service glance] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:41.244258 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206efc9b-af05-4da1-a108-73e2451f1a48 service glance] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:41.244630 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206efc9b-af05-4da1-a108-73e2451f1a48 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:41.266681 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206efc9b-af05-4da1-a108-73e2451f1a48 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:41.268479 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206efc9b-af05-4da1-a108-73e2451f1a48 service glance] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:41.281745 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1032/2031] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:45.876137 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a418abcb-ee0f-41a6-9d46-b33185fa44f5 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:45.942381 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a418abcb-ee0f-41a6-9d46-b33185fa44f5 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:45.946872 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91302255-3271-4c9b-856c-ba510ad7a4cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:45.947456 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91302255-3271-4c9b-856c-ba510ad7a4cb service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:45.947737 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91302255-3271-4c9b-856c-ba510ad7a4cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:45.948013 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91302255-3271-4c9b-856c-ba510ad7a4cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:46.039690 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91302255-3271-4c9b-856c-ba510ad7a4cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:46.041340 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91302255-3271-4c9b-856c-ba510ad7a4cb service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:46.053232 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1000/2032] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:45 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:51.276094 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=80,keystone:INSERT=5 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:54.284442 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206efc9b-af05-4da1-a108-73e2451f1a48 service glance] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:54.374523 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206efc9b-af05-4da1-a108-73e2451f1a48 service glance] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:54.380363 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999a8622-bc86-4777-a9c0-09e3786d7c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:54.381673 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a8622-bc86-4777-a9c0-09e3786d7c4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:54.381995 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a8622-bc86-4777-a9c0-09e3786d7c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:54.382276 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a8622-bc86-4777-a9c0-09e3786d7c4d admin admin] PATH_INFO: `/v3/users/200f52cbd2154662880ddf6292a2e9b4` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:54.383544 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999a8622-bc86-4777-a9c0-09e3786d7c4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=200f52cbd2154662880ddf6292a2e9b4)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:54.385539 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999a8622-bc86-4777-a9c0-09e3786d7c4d admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:54.434555 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-999a8622-bc86-4777-a9c0-09e3786d7c4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '200f52cbd2154662880ddf6292a2e9b4'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:54.447211 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-999a8622-bc86-4777-a9c0-09e3786d7c4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '200f52cbd2154662880ddf6292a2e9b4'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:54.456946 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-999a8622-bc86-4777-a9c0-09e3786d7c4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '200f52cbd2154662880ddf6292a2e9b4'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:54.457786 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-999a8622-bc86-4777-a9c0-09e3786d7c4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '200f52cbd2154662880ddf6292a2e9b4'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:54.458668 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-999a8622-bc86-4777-a9c0-09e3786d7c4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '200f52cbd2154662880ddf6292a2e9b4'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:54.471642 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1033/2033] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:41:54 2022] DELETE /identity/v3/users/200f52cbd2154662880ddf6292a2e9b4 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:54.476298 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caea9f7-2ac3-4130-a3ce-9ff714d38074 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:54.476833 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caea9f7-2ac3-4130-a3ce-9ff714d38074 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:54.476833 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caea9f7-2ac3-4130-a3ce-9ff714d38074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:54.495058 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-4caea9f7-2ac3-4130-a3ce-9ff714d38074 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:54.496273 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-4caea9f7-2ac3-4130-a3ce-9ff714d38074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:54.532415 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:54.568086 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1001/2034] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:41:54.576734 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999a8622-bc86-4777-a9c0-09e3786d7c4d admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:54.578302 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-999a8622-bc86-4777-a9c0-09e3786d7c4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:54.642574 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999a8622-bc86-4777-a9c0-09e3786d7c4d admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:54.647453 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7ce7d5-3b30-4c24-bf12-94949014f398 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:54.648132 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ce7d5-3b30-4c24-bf12-94949014f398 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:54.648664 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ce7d5-3b30-4c24-bf12-94949014f398 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:54.648973 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ce7d5-3b30-4c24-bf12-94949014f398 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:54.659560 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7ce7d5-3b30-4c24-bf12-94949014f398 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:54.661327 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7ce7d5-3b30-4c24-bf12-94949014f398 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:54.672071 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1034/2035] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:55.283922 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caea9f7-2ac3-4130-a3ce-9ff714d38074 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:55.290126 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-4caea9f7-2ac3-4130-a3ce-9ff714d38074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:55.304787 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caea9f7-2ac3-4130-a3ce-9ff714d38074 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:55.314156 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3999ed26-ebc9-4647-bb8f-880b8da17622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:55.315071 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3999ed26-ebc9-4647-bb8f-880b8da17622 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:55.315363 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3999ed26-ebc9-4647-bb8f-880b8da17622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:55.315939 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3999ed26-ebc9-4647-bb8f-880b8da17622 admin admin] PATH_INFO: `/v3/projects/3bba6f91e5b34f2992f9aabf1a3725bf` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:55.317146 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3999ed26-ebc9-4647-bb8f-880b8da17622 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bba6f91e5b34f2992f9aabf1a3725bf)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:55.319009 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3999ed26-ebc9-4647-bb8f-880b8da17622 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:55.384045 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-3999ed26-ebc9-4647-bb8f-880b8da17622 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bba6f91e5b34f2992f9aabf1a3725bf', 'request_id': 'req-3999ed26-ebc9-4647-bb8f-880b8da17622'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:55.391609 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-3999ed26-ebc9-4647-bb8f-880b8da17622 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bba6f91e5b34f2992f9aabf1a3725bf', 'request_id': 'req-3999ed26-ebc9-4647-bb8f-880b8da17622'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:55.398424 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-3999ed26-ebc9-4647-bb8f-880b8da17622 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bba6f91e5b34f2992f9aabf1a3725bf', 'request_id': 'req-3999ed26-ebc9-4647-bb8f-880b8da17622'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:55.433397 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-3999ed26-ebc9-4647-bb8f-880b8da17622 admin admin] The token cache is being invalidate because project 3bba6f91e5b34f2992f9aabf1a3725bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:41:55.433739 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-3999ed26-ebc9-4647-bb8f-880b8da17622 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:55.437238 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-3999ed26-ebc9-4647-bb8f-880b8da17622 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:55.442925 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1002/2036] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:41:55 2022] DELETE /identity/v3/projects/3bba6f91e5b34f2992f9aabf1a3725bf => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:55.462834 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7ce7d5-3b30-4c24-bf12-94949014f398 service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:55.470033 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7ce7d5-3b30-4c24-bf12-94949014f398 service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:55.501312 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:55.575264 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42801605-b237-421a-a28b-a30f915f9051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:55.576574 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42801605-b237-421a-a28b-a30f915f9051 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:55.576920 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42801605-b237-421a-a28b-a30f915f9051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:55.577262 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42801605-b237-421a-a28b-a30f915f9051 admin admin] PATH_INFO: `/v3/users/93ed8954ab3149fba2a7749893e7f13b` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:55.578664 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42801605-b237-421a-a28b-a30f915f9051 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93ed8954ab3149fba2a7749893e7f13b)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:55.580696 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42801605-b237-421a-a28b-a30f915f9051 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:55.643150 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-42801605-b237-421a-a28b-a30f915f9051 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93ed8954ab3149fba2a7749893e7f13b'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:55.666125 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-42801605-b237-421a-a28b-a30f915f9051 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93ed8954ab3149fba2a7749893e7f13b'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:55.673988 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-42801605-b237-421a-a28b-a30f915f9051 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93ed8954ab3149fba2a7749893e7f13b'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:55.674615 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-42801605-b237-421a-a28b-a30f915f9051 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93ed8954ab3149fba2a7749893e7f13b'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:55.675507 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-42801605-b237-421a-a28b-a30f915f9051 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93ed8954ab3149fba2a7749893e7f13b'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:55.688303 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1035/2037] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:41:55 2022] DELETE /identity/v3/users/93ed8954ab3149fba2a7749893e7f13b => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:55.977620 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3999ed26-ebc9-4647-bb8f-880b8da17622 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:55.979245 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-3999ed26-ebc9-4647-bb8f-880b8da17622 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:55.995523 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3999ed26-ebc9-4647-bb8f-880b8da17622 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:56.076578 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fa8447-0869-4b94-8298-5596fdaa8348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:56.077479 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fa8447-0869-4b94-8298-5596fdaa8348 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:56.077789 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fa8447-0869-4b94-8298-5596fdaa8348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:56.078070 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fa8447-0869-4b94-8298-5596fdaa8348 admin admin] PATH_INFO: `/v3/projects/cec7ff6a1abf415c9a2f50c73d0a44f3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:56.079357 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fa8447-0869-4b94-8298-5596fdaa8348 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cec7ff6a1abf415c9a2f50c73d0a44f3)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:56.081303 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fa8447-0869-4b94-8298-5596fdaa8348 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:56.135362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-73fa8447-0869-4b94-8298-5596fdaa8348 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cec7ff6a1abf415c9a2f50c73d0a44f3', 'request_id': 'req-73fa8447-0869-4b94-8298-5596fdaa8348'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:56.142321 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-73fa8447-0869-4b94-8298-5596fdaa8348 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cec7ff6a1abf415c9a2f50c73d0a44f3', 'request_id': 'req-73fa8447-0869-4b94-8298-5596fdaa8348'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:56.151388 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-73fa8447-0869-4b94-8298-5596fdaa8348 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cec7ff6a1abf415c9a2f50c73d0a44f3', 'request_id': 'req-73fa8447-0869-4b94-8298-5596fdaa8348'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:56.153251 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:56.170519 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-73fa8447-0869-4b94-8298-5596fdaa8348 admin admin] The token cache is being invalidate because project cec7ff6a1abf415c9a2f50c73d0a44f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:41:56.170975 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-73fa8447-0869-4b94-8298-5596fdaa8348 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:56.172477 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-73fa8447-0869-4b94-8298-5596fdaa8348 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:56.175980 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1003/2038] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:41:55 2022] DELETE /identity/v3/projects/cec7ff6a1abf415c9a2f50c73d0a44f3 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:56.183690 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58096ea3-04f9-400a-907d-09ad3be88a0e None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:56.183985 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58096ea3-04f9-400a-907d-09ad3be88a0e None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:56.184266 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58096ea3-04f9-400a-907d-09ad3be88a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:56.214630 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-58096ea3-04f9-400a-907d-09ad3be88a0e None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:56.216032 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-58096ea3-04f9-400a-907d-09ad3be88a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:56.320109 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1036/2039] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:56.329311 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2647dabd-e1a8-40c0-a4a2-2d322d57b39a None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:56.329918 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2647dabd-e1a8-40c0-a4a2-2d322d57b39a None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:56.330352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2647dabd-e1a8-40c0-a4a2-2d322d57b39a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:56.350556 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-2647dabd-e1a8-40c0-a4a2-2d322d57b39a None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:56.357942 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-2647dabd-e1a8-40c0-a4a2-2d322d57b39a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:56.367445 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1004/2040] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:56.371964 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58096ea3-04f9-400a-907d-09ad3be88a0e None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:56.384894 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58096ea3-04f9-400a-907d-09ad3be88a0e None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:56.403374 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e813b4-6a79-47a8-87f2-1433c385fb1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:56.404169 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e813b4-6a79-47a8-87f2-1433c385fb1e admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:56.404424 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e813b4-6a79-47a8-87f2-1433c385fb1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:56.410435 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e813b4-6a79-47a8-87f2-1433c385fb1e admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:56.410435 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e813b4-6a79-47a8-87f2-1433c385fb1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:41:56.410435 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e813b4-6a79-47a8-87f2-1433c385fb1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:56.418384 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e813b4-6a79-47a8-87f2-1433c385fb1e admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:56.441730 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1037/2041] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:41:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:41:56.445040 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2647dabd-e1a8-40c0-a4a2-2d322d57b39a None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:56.455734 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2647dabd-e1a8-40c0-a4a2-2d322d57b39a None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:56.460749 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0020daf9-96f8-43bf-8fe9-b292d4ab5570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:56.461457 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0020daf9-96f8-43bf-8fe9-b292d4ab5570 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:56.461912 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0020daf9-96f8-43bf-8fe9-b292d4ab5570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:56.462026 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0020daf9-96f8-43bf-8fe9-b292d4ab5570 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:56.462717 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0020daf9-96f8-43bf-8fe9-b292d4ab5570 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:56.464642 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0020daf9-96f8-43bf-8fe9-b292d4ab5570 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:56.499723 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1005/2042] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:41:56 2022] POST /identity/v3/projects => generated 397 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:56.503871 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e813b4-6a79-47a8-87f2-1433c385fb1e admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:56.515369 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e813b4-6a79-47a8-87f2-1433c385fb1e admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:56.554919 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:56.593266 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caea7dfa-6d59-4100-9d89-83d7337baad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:56.594259 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caea7dfa-6d59-4100-9d89-83d7337baad0 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:56.594539 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caea7dfa-6d59-4100-9d89-83d7337baad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:56.594810 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caea7dfa-6d59-4100-9d89-83d7337baad0 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:56.595500 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caea7dfa-6d59-4100-9d89-83d7337baad0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:56.598331 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caea7dfa-6d59-4100-9d89-83d7337baad0 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:56.620954 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1038/2043] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:41:56 2022] POST /identity/v3/users => generated 299 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:56.625895 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0020daf9-96f8-43bf-8fe9-b292d4ab5570 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:56.637875 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0020daf9-96f8-43bf-8fe9-b292d4ab5570 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:56.642560 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928dbb41-0580-4661-9ec0-c9ff7a9b4304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:56.643228 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928dbb41-0580-4661-9ec0-c9ff7a9b4304 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:56.643555 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928dbb41-0580-4661-9ec0-c9ff7a9b4304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:56.643930 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928dbb41-0580-4661-9ec0-c9ff7a9b4304 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:56.644667 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928dbb41-0580-4661-9ec0-c9ff7a9b4304 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:56.646307 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928dbb41-0580-4661-9ec0-c9ff7a9b4304 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:56.658113 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1006/2044] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:56.662762 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caea7dfa-6d59-4100-9d89-83d7337baad0 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:56.679448 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caea7dfa-6d59-4100-9d89-83d7337baad0 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:56.684762 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1734cd-818a-48cd-86f3-4873ed57e3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:56.685935 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1734cd-818a-48cd-86f3-4873ed57e3cb admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:56.685935 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1734cd-818a-48cd-86f3-4873ed57e3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:56.686247 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1734cd-818a-48cd-86f3-4873ed57e3cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:56.687052 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1734cd-818a-48cd-86f3-4873ed57e3cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:56.688733 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1734cd-818a-48cd-86f3-4873ed57e3cb admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:56.701449 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1039/2045] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:56 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:56.706047 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928dbb41-0580-4661-9ec0-c9ff7a9b4304 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:56.723756 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928dbb41-0580-4661-9ec0-c9ff7a9b4304 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:56.728917 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa7f41-ef83-4aaf-a778-e0276deb4306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:56.729851 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa7f41-ef83-4aaf-a778-e0276deb4306 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:56.730137 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa7f41-ef83-4aaf-a778-e0276deb4306 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:56.730418 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa7f41-ef83-4aaf-a778-e0276deb4306 admin admin] PATH_INFO: `/v3/projects/b693f4e621bb4ffc848e598e7efa9559/users/a790e4d2039942fdbbcd9eae9d99b854/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:56.756252 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aa7f41-ef83-4aaf-a778-e0276deb4306 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b693f4e621bb4ffc848e598e7efa9559, user_id=a790e4d2039942fdbbcd9eae9d99b854, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:56.756252 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aa7f41-ef83-4aaf-a778-e0276deb4306 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:56.779990 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1007/2046] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:56 2022] PUT /identity/v3/projects/b693f4e621bb4ffc848e598e7efa9559/users/a790e4d2039942fdbbcd9eae9d99b854/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:41:56.792971 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b26049-2cda-46f3-a9f0-5ca02a201fbf None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:56.793311 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b26049-2cda-46f3-a9f0-5ca02a201fbf None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:56.793562 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b26049-2cda-46f3-a9f0-5ca02a201fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:56.837948 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-b9b26049-2cda-46f3-a9f0-5ca02a201fbf None None] MFA Rules not processed for user `a790e4d2039942fdbbcd9eae9d99b854`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:56.839175 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b9b26049-2cda-46f3-a9f0-5ca02a201fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:56.936307 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1040/2047] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:56 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:56.952754 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa7f41-ef83-4aaf-a778-e0276deb4306 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:56.954995 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-b0aa7f41-ef83-4aaf-a778-e0276deb4306 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:57.038625 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa7f41-ef83-4aaf-a778-e0276deb4306 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:57.043624 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098adae6-cefa-4582-b6a9-ff3760311505 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:57.044535 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098adae6-cefa-4582-b6a9-ff3760311505 service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:57.044893 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098adae6-cefa-4582-b6a9-ff3760311505 service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:57.045237 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098adae6-cefa-4582-b6a9-ff3760311505 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:57.056036 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098adae6-cefa-4582-b6a9-ff3760311505 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:57.057899 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098adae6-cefa-4582-b6a9-ff3760311505 service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:57.086769 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1008/2048] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:56 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:57.248718 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b26049-2cda-46f3-a9f0-5ca02a201fbf None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:57.250718 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b9b26049-2cda-46f3-a9f0-5ca02a201fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:57.357406 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b26049-2cda-46f3-a9f0-5ca02a201fbf None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:57.373580 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2fcb2b-a1f2-4ca1-9607-61344130e981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:57.373580 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2fcb2b-a1f2-4ca1-9607-61344130e981 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:57.373580 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2fcb2b-a1f2-4ca1-9607-61344130e981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:57.373580 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2fcb2b-a1f2-4ca1-9607-61344130e981 admin admin] PATH_INFO: `/v3/users/9a304684b4a9402398c08f9746fcc3da` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:57.373984 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2fcb2b-a1f2-4ca1-9607-61344130e981 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a304684b4a9402398c08f9746fcc3da)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:57.378472 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2fcb2b-a1f2-4ca1-9607-61344130e981 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:57.433261 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:57.443692 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0d2fcb2b-a1f2-4ca1-9607-61344130e981 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a304684b4a9402398c08f9746fcc3da'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:57.479833 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0d2fcb2b-a1f2-4ca1-9607-61344130e981 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a304684b4a9402398c08f9746fcc3da'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:57.495082 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0d2fcb2b-a1f2-4ca1-9607-61344130e981 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a304684b4a9402398c08f9746fcc3da'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:57.495714 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0d2fcb2b-a1f2-4ca1-9607-61344130e981 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a304684b4a9402398c08f9746fcc3da'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:57.496410 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0d2fcb2b-a1f2-4ca1-9607-61344130e981 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a304684b4a9402398c08f9746fcc3da'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:57.516055 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1041/2049] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:41:57 2022] DELETE /identity/v3/users/9a304684b4a9402398c08f9746fcc3da => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:57.520913 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35034a2-051c-43d0-b911-5b9d251ff7fc None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:57.521414 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35034a2-051c-43d0-b911-5b9d251ff7fc None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:57.521715 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35034a2-051c-43d0-b911-5b9d251ff7fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:57.542897 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-a35034a2-051c-43d0-b911-5b9d251ff7fc None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:57.544382 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-a35034a2-051c-43d0-b911-5b9d251ff7fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:57.628860 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1009/2050] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:57.640396 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2fcb2b-a1f2-4ca1-9607-61344130e981 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:57.640396 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-0d2fcb2b-a1f2-4ca1-9607-61344130e981 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:57.703785 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2fcb2b-a1f2-4ca1-9607-61344130e981 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:57.708422 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab2c4ee-a7ca-47da-a1b9-706d27c4b710 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:57.709085 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab2c4ee-a7ca-47da-a1b9-706d27c4b710 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:57.709372 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab2c4ee-a7ca-47da-a1b9-706d27c4b710 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:57.709829 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab2c4ee-a7ca-47da-a1b9-706d27c4b710 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:57.719611 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab2c4ee-a7ca-47da-a1b9-706d27c4b710 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:57.721473 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab2c4ee-a7ca-47da-a1b9-706d27c4b710 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:57.732589 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1042/2051] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:58.009308 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35034a2-051c-43d0-b911-5b9d251ff7fc None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:58.010935 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-a35034a2-051c-43d0-b911-5b9d251ff7fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:58.024579 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35034a2-051c-43d0-b911-5b9d251ff7fc None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:58.031583 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f026b480-f219-434a-a651-420f48a52638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:58.032412 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f026b480-f219-434a-a651-420f48a52638 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:58.033318 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f026b480-f219-434a-a651-420f48a52638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:58.034604 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f026b480-f219-434a-a651-420f48a52638 admin admin] PATH_INFO: `/v3/projects/83875d4e665744acb2e19e07608d79c3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:58.037033 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f026b480-f219-434a-a651-420f48a52638 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83875d4e665744acb2e19e07608d79c3)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:58.039179 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f026b480-f219-434a-a651-420f48a52638 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:58.072284 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:INSERT=3,keystone:DELETE=2,keystone:SELECT=95 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:58.090837 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f026b480-f219-434a-a651-420f48a52638 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83875d4e665744acb2e19e07608d79c3', 'request_id': 'req-f026b480-f219-434a-a651-420f48a52638'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:58.098234 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f026b480-f219-434a-a651-420f48a52638 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83875d4e665744acb2e19e07608d79c3', 'request_id': 'req-f026b480-f219-434a-a651-420f48a52638'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:58.103887 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f026b480-f219-434a-a651-420f48a52638 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83875d4e665744acb2e19e07608d79c3', 'request_id': 'req-f026b480-f219-434a-a651-420f48a52638'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:58.119216 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f026b480-f219-434a-a651-420f48a52638 admin admin] The token cache is being invalidate because project 83875d4e665744acb2e19e07608d79c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:41:58.119502 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f026b480-f219-434a-a651-420f48a52638 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:58.120124 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f026b480-f219-434a-a651-420f48a52638 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:58.121676 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1010/2052] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:41:58 2022] DELETE /identity/v3/projects/83875d4e665744acb2e19e07608d79c3 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:58.126267 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab2c4ee-a7ca-47da-a1b9-706d27c4b710 service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:58.136994 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab2c4ee-a7ca-47da-a1b9-706d27c4b710 service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:58.195123 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8b1a18-ba33-43f7-97a9-bbdcbcaf6554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:58.196237 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b1a18-ba33-43f7-97a9-bbdcbcaf6554 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:58.196520 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b1a18-ba33-43f7-97a9-bbdcbcaf6554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:58.196794 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b1a18-ba33-43f7-97a9-bbdcbcaf6554 admin admin] PATH_INFO: `/v3/users/14ae5910834940c3b69d2bdd64dad683` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:58.198139 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8b1a18-ba33-43f7-97a9-bbdcbcaf6554 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14ae5910834940c3b69d2bdd64dad683)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:58.201420 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8b1a18-ba33-43f7-97a9-bbdcbcaf6554 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:58.251060 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7b8b1a18-ba33-43f7-97a9-bbdcbcaf6554 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14ae5910834940c3b69d2bdd64dad683'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:58.261615 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:58.264216 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7b8b1a18-ba33-43f7-97a9-bbdcbcaf6554 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14ae5910834940c3b69d2bdd64dad683'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:58.272784 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7b8b1a18-ba33-43f7-97a9-bbdcbcaf6554 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14ae5910834940c3b69d2bdd64dad683'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:58.273765 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7b8b1a18-ba33-43f7-97a9-bbdcbcaf6554 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14ae5910834940c3b69d2bdd64dad683'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:58.274400 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7b8b1a18-ba33-43f7-97a9-bbdcbcaf6554 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14ae5910834940c3b69d2bdd64dad683'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:58.287832 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1043/2053] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:41:58 2022] DELETE /identity/v3/users/14ae5910834940c3b69d2bdd64dad683 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:58.611770 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f026b480-f219-434a-a651-420f48a52638 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:58.613417 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-f026b480-f219-434a-a651-420f48a52638 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:58.643185 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f026b480-f219-434a-a651-420f48a52638 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:58.733836 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e108271-749f-4fd2-804d-5adcafece77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:58.733836 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e108271-749f-4fd2-804d-5adcafece77b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:58.733836 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e108271-749f-4fd2-804d-5adcafece77b admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:58.733836 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e108271-749f-4fd2-804d-5adcafece77b admin admin] PATH_INFO: `/v3/projects/09cd5e4f1b0746f692106b7fc6683b51` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:58.734175 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e108271-749f-4fd2-804d-5adcafece77b admin admin] RBAC: Authorizing `identity:delete_project(project_id=09cd5e4f1b0746f692106b7fc6683b51)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:58.734175 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e108271-749f-4fd2-804d-5adcafece77b admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:58.789303 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-3e108271-749f-4fd2-804d-5adcafece77b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09cd5e4f1b0746f692106b7fc6683b51', 'request_id': 'req-3e108271-749f-4fd2-804d-5adcafece77b'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:58.797627 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-3e108271-749f-4fd2-804d-5adcafece77b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09cd5e4f1b0746f692106b7fc6683b51', 'request_id': 'req-3e108271-749f-4fd2-804d-5adcafece77b'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:58.805231 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-3e108271-749f-4fd2-804d-5adcafece77b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09cd5e4f1b0746f692106b7fc6683b51', 'request_id': 'req-3e108271-749f-4fd2-804d-5adcafece77b'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:58.822010 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-3e108271-749f-4fd2-804d-5adcafece77b admin admin] The token cache is being invalidate because project 09cd5e4f1b0746f692106b7fc6683b51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:41:58.822372 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-3e108271-749f-4fd2-804d-5adcafece77b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:58.823254 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-3e108271-749f-4fd2-804d-5adcafece77b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:58.827376 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1011/2054] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:41:58 2022] DELETE /identity/v3/projects/09cd5e4f1b0746f692106b7fc6683b51 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:58.834028 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aedfd7-ccce-45ec-8b73-e574d906dc0f None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:58.834486 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aedfd7-ccce-45ec-8b73-e574d906dc0f None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:58.834711 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aedfd7-ccce-45ec-8b73-e574d906dc0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:58.854771 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-42aedfd7-ccce-45ec-8b73-e574d906dc0f None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:58.855996 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-42aedfd7-ccce-45ec-8b73-e574d906dc0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:58.937647 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1044/2055] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:58.949145 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d662dd-8f50-4787-92b3-b22d69213e66 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:58.949145 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d662dd-8f50-4787-92b3-b22d69213e66 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:58.949316 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d662dd-8f50-4787-92b3-b22d69213e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:58.968684 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-a2d662dd-8f50-4787-92b3-b22d69213e66 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:58.969950 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-a2d662dd-8f50-4787-92b3-b22d69213e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:58.979300 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1012/2056] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:41:58.983643 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aedfd7-ccce-45ec-8b73-e574d906dc0f None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:58.994337 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aedfd7-ccce-45ec-8b73-e574d906dc0f None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:59.001043 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d270490-4063-44be-81b8-ecbdcdd32289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:59.002114 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d270490-4063-44be-81b8-ecbdcdd32289 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:59.002494 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d270490-4063-44be-81b8-ecbdcdd32289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:59.003645 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d270490-4063-44be-81b8-ecbdcdd32289 admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:59.004359 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d270490-4063-44be-81b8-ecbdcdd32289 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:41:59.004676 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d270490-4063-44be-81b8-ecbdcdd32289 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:59.006372 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d270490-4063-44be-81b8-ecbdcdd32289 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:59.019456 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1045/2057] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:41:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:41:59.023793 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d662dd-8f50-4787-92b3-b22d69213e66 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:59.033173 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d662dd-8f50-4787-92b3-b22d69213e66 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:59.038387 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc320fe1-8525-4175-82b8-529d4c002270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:59.039041 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc320fe1-8525-4175-82b8-529d4c002270 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:59.039329 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc320fe1-8525-4175-82b8-529d4c002270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:59.039870 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc320fe1-8525-4175-82b8-529d4c002270 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:59.040287 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc320fe1-8525-4175-82b8-529d4c002270 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:59.042275 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc320fe1-8525-4175-82b8-529d4c002270 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:59.070236 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1013/2058] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:41:59 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:59.075027 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d270490-4063-44be-81b8-ecbdcdd32289 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:59.087406 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d270490-4063-44be-81b8-ecbdcdd32289 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:59.153357 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef7204a-7641-4e9f-8cef-41581c61b52f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:59.153977 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef7204a-7641-4e9f-8cef-41581c61b52f admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:59.154270 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef7204a-7641-4e9f-8cef-41581c61b52f admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:59.154553 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef7204a-7641-4e9f-8cef-41581c61b52f admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:59.155265 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef7204a-7641-4e9f-8cef-41581c61b52f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:59.157192 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef7204a-7641-4e9f-8cef-41581c61b52f admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:59.181678 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1046/2059] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:41:59 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:59.185755 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc320fe1-8525-4175-82b8-529d4c002270 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:59.197036 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc320fe1-8525-4175-82b8-529d4c002270 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:59.201786 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bb4dcd-d959-4dda-b116-50441eedc1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:59.202460 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb4dcd-d959-4dda-b116-50441eedc1d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:59.202742 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb4dcd-d959-4dda-b116-50441eedc1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:59.203021 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb4dcd-d959-4dda-b116-50441eedc1d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:59.203772 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bb4dcd-d959-4dda-b116-50441eedc1d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:59.205381 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bb4dcd-d959-4dda-b116-50441eedc1d1 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:59.217324 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1014/2060] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:59.221273 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef7204a-7641-4e9f-8cef-41581c61b52f admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:59.231941 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef7204a-7641-4e9f-8cef-41581c61b52f admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:59.237737 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fed383e-a4f8-4a42-95ab-5da8f6233eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:59.238634 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed383e-a4f8-4a42-95ab-5da8f6233eac admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:59.238826 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed383e-a4f8-4a42-95ab-5da8f6233eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:59.239116 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed383e-a4f8-4a42-95ab-5da8f6233eac admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:59.239854 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fed383e-a4f8-4a42-95ab-5da8f6233eac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:59.241471 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fed383e-a4f8-4a42-95ab-5da8f6233eac admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:59.253542 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1047/2061] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:59.258230 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bb4dcd-d959-4dda-b116-50441eedc1d1 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:59.265054 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:59.269124 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bb4dcd-d959-4dda-b116-50441eedc1d1 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:59.276028 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3034520-91b5-4928-9512-ac468748d9e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:59.277090 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3034520-91b5-4928-9512-ac468748d9e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:59.286181 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3034520-91b5-4928-9512-ac468748d9e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:59.286181 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3034520-91b5-4928-9512-ac468748d9e0 admin admin] PATH_INFO: `/v3/projects/9e75e5e1ff1940ae8991a776a6c37699/users/98324e11664c42c28d6eff3342802581/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:59.293263 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3034520-91b5-4928-9512-ac468748d9e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e75e5e1ff1940ae8991a776a6c37699, user_id=98324e11664c42c28d6eff3342802581, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:59.295943 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3034520-91b5-4928-9512-ac468748d9e0 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:59.309376 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1015/2062] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:59 2022] PUT /identity/v3/projects/9e75e5e1ff1940ae8991a776a6c37699/users/98324e11664c42c28d6eff3342802581/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:59.325905 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b882c-c426-434c-8883-30445e2025c8 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:59.326186 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b882c-c426-434c-8883-30445e2025c8 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:59.326460 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b882c-c426-434c-8883-30445e2025c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:59.359682 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-520b882c-c426-434c-8883-30445e2025c8 None None] MFA Rules not processed for user `98324e11664c42c28d6eff3342802581`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:59.360963 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-520b882c-c426-434c-8883-30445e2025c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:59.395238 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:59.446769 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1048/2063] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:59.451172 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3034520-91b5-4928-9512-ac468748d9e0 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:59.514321 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3034520-91b5-4928-9512-ac468748d9e0 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:59.519464 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be787c-c1cd-4444-a473-ea0b5f2acff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:59.520134 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be787c-c1cd-4444-a473-ea0b5f2acff1 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:59.520424 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be787c-c1cd-4444-a473-ea0b5f2acff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:59.520704 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be787c-c1cd-4444-a473-ea0b5f2acff1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:59.521484 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9be787c-c1cd-4444-a473-ea0b5f2acff1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:59.523467 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9be787c-c1cd-4444-a473-ea0b5f2acff1 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:59.550877 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1016/2064] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:41:59 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:41:59.555102 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520b882c-c426-434c-8883-30445e2025c8 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:59.566330 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520b882c-c426-434c-8883-30445e2025c8 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:59.623838 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f780e327-1d54-43f2-9acb-31be97019192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:59.624514 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f780e327-1d54-43f2-9acb-31be97019192 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:59.624801 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f780e327-1d54-43f2-9acb-31be97019192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:59.625080 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f780e327-1d54-43f2-9acb-31be97019192 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:59.625819 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f780e327-1d54-43f2-9acb-31be97019192 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:59.628420 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f780e327-1d54-43f2-9acb-31be97019192 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:59.655808 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1049/2065] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:41:59 2022] POST /identity/v3/users => generated 299 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:59.660937 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be787c-c1cd-4444-a473-ea0b5f2acff1 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:59.671675 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be787c-c1cd-4444-a473-ea0b5f2acff1 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:59.677572 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c57cc5c-7724-469c-bc12-fddacc630704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:59.678268 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c57cc5c-7724-469c-bc12-fddacc630704 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:59.678555 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c57cc5c-7724-469c-bc12-fddacc630704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:59.678838 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c57cc5c-7724-469c-bc12-fddacc630704 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:59.679559 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c57cc5c-7724-469c-bc12-fddacc630704 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:59.681198 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c57cc5c-7724-469c-bc12-fddacc630704 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:59.696046 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1017/2066] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:59.697640 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f780e327-1d54-43f2-9acb-31be97019192 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:59.709287 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f780e327-1d54-43f2-9acb-31be97019192 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:59.714125 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6b0e96-2f36-4e92-8833-3480ee0da83d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:59.715022 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6b0e96-2f36-4e92-8833-3480ee0da83d admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:59.715304 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6b0e96-2f36-4e92-8833-3480ee0da83d admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:59.715581 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6b0e96-2f36-4e92-8833-3480ee0da83d admin admin] PATH_INFO: `/v3/projects/dbb8d829098c47b0a3c46b010ac2fd17/users/97334c4867e5485a98e19e0e98063b28/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:59.734850 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6b0e96-2f36-4e92-8833-3480ee0da83d admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbb8d829098c47b0a3c46b010ac2fd17, user_id=97334c4867e5485a98e19e0e98063b28, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:59.738441 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6b0e96-2f36-4e92-8833-3480ee0da83d admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:59.752434 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1050/2067] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:59 2022] PUT /identity/v3/projects/dbb8d829098c47b0a3c46b010ac2fd17/users/97334c4867e5485a98e19e0e98063b28/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:59.756535 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c57cc5c-7724-469c-bc12-fddacc630704 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:59.767388 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c57cc5c-7724-469c-bc12-fddacc630704 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:59.834156 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5524657c-0d60-4488-84d2-eec431d8725e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:59.834835 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524657c-0d60-4488-84d2-eec431d8725e admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:59.835149 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524657c-0d60-4488-84d2-eec431d8725e admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:59.835426 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524657c-0d60-4488-84d2-eec431d8725e admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:59.836164 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5524657c-0d60-4488-84d2-eec431d8725e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:59.837785 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5524657c-0d60-4488-84d2-eec431d8725e admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:59.858957 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1018/2068] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:59 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:59.859932 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6b0e96-2f36-4e92-8833-3480ee0da83d admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:59.871315 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6b0e96-2f36-4e92-8833-3480ee0da83d admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:59.877556 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c738664a-1e5f-46ad-917e-e66171e31857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:59.879234 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c738664a-1e5f-46ad-917e-e66171e31857 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:59.879527 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c738664a-1e5f-46ad-917e-e66171e31857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:59.879949 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c738664a-1e5f-46ad-917e-e66171e31857 admin admin] PATH_INFO: `/v3/projects/dbb8d829098c47b0a3c46b010ac2fd17/users/97334c4867e5485a98e19e0e98063b28/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:59.882514 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c738664a-1e5f-46ad-917e-e66171e31857 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbb8d829098c47b0a3c46b010ac2fd17, user_id=97334c4867e5485a98e19e0e98063b28, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:59.885415 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c738664a-1e5f-46ad-917e-e66171e31857 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:59.897015 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1051/2069] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:59 2022] PUT /identity/v3/projects/dbb8d829098c47b0a3c46b010ac2fd17/users/97334c4867e5485a98e19e0e98063b28/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:41:59.908738 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35aca69-cb3d-47bd-8516-c40575f84505 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:59.909408 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35aca69-cb3d-47bd-8516-c40575f84505 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:59.909754 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35aca69-cb3d-47bd-8516-c40575f84505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:59.938676 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-d35aca69-cb3d-47bd-8516-c40575f84505 None None] MFA Rules not processed for user `97334c4867e5485a98e19e0e98063b28`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:59.941376 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d35aca69-cb3d-47bd-8516-c40575f84505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:00.007503 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:00.044407 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1019/2070] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:41:59 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:00.056045 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c738664a-1e5f-46ad-917e-e66171e31857 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:00.057802 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-c738664a-1e5f-46ad-917e-e66171e31857 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:00.127787 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c738664a-1e5f-46ad-917e-e66171e31857 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:00.131672 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89858147-4ee6-4b37-9e32-4d8c954732fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:00.132327 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89858147-4ee6-4b37-9e32-4d8c954732fd service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:00.132626 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89858147-4ee6-4b37-9e32-4d8c954732fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:00.132896 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89858147-4ee6-4b37-9e32-4d8c954732fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:00.146216 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89858147-4ee6-4b37-9e32-4d8c954732fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:00.148093 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89858147-4ee6-4b37-9e32-4d8c954732fd service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:00.160022 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1052/2071] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:00 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:42:00.239763 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35aca69-cb3d-47bd-8516-c40575f84505 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:00.324334 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35aca69-cb3d-47bd-8516-c40575f84505 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:00.329226 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7ee4f3-d205-47dc-9316-eaa55e58cb27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:00.330144 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ee4f3-d205-47dc-9316-eaa55e58cb27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:00.330427 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ee4f3-d205-47dc-9316-eaa55e58cb27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:00.330703 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ee4f3-d205-47dc-9316-eaa55e58cb27 admin admin] PATH_INFO: `/v3/users/98324e11664c42c28d6eff3342802581` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:00.333091 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7ee4f3-d205-47dc-9316-eaa55e58cb27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98324e11664c42c28d6eff3342802581)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:00.335588 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7ee4f3-d205-47dc-9316-eaa55e58cb27 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:00.401122 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-2d7ee4f3-d205-47dc-9316-eaa55e58cb27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98324e11664c42c28d6eff3342802581'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:00.418384 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-2d7ee4f3-d205-47dc-9316-eaa55e58cb27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98324e11664c42c28d6eff3342802581'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:00.419220 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-2d7ee4f3-d205-47dc-9316-eaa55e58cb27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98324e11664c42c28d6eff3342802581'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:00.428891 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-2d7ee4f3-d205-47dc-9316-eaa55e58cb27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98324e11664c42c28d6eff3342802581'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:00.429645 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-2d7ee4f3-d205-47dc-9316-eaa55e58cb27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98324e11664c42c28d6eff3342802581'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:00.446373 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1020/2072] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:42:00 2022] DELETE /identity/v3/users/98324e11664c42c28d6eff3342802581 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:00.451203 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f071f4-63cb-49ef-914b-b9388e814183 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:00.451495 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f071f4-63cb-49ef-914b-b9388e814183 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:00.451777 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f071f4-63cb-49ef-914b-b9388e814183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:00.479322 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-c2f071f4-63cb-49ef-914b-b9388e814183 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:00.479322 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-c2f071f4-63cb-49ef-914b-b9388e814183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:00.507445 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:00.548213 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1053/2073] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:42:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:42:00.558712 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7ee4f3-d205-47dc-9316-eaa55e58cb27 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:00.560742 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-2d7ee4f3-d205-47dc-9316-eaa55e58cb27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:00.628909 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7ee4f3-d205-47dc-9316-eaa55e58cb27 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:00.632837 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489a1bea-566c-45fb-ac83-add657f1d87c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:00.633607 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a1bea-566c-45fb-ac83-add657f1d87c service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:00.633918 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a1bea-566c-45fb-ac83-add657f1d87c service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:00.634195 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a1bea-566c-45fb-ac83-add657f1d87c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:00.644829 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489a1bea-566c-45fb-ac83-add657f1d87c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:00.646507 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489a1bea-566c-45fb-ac83-add657f1d87c service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:00.650695 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:00.657820 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1021/2074] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:42:01.039361 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f071f4-63cb-49ef-914b-b9388e814183 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:01.041105 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-c2f071f4-63cb-49ef-914b-b9388e814183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:01.055203 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f071f4-63cb-49ef-914b-b9388e814183 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:01.067216 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a0e019-17db-4d28-9078-7d12e53a6c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:01.068156 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a0e019-17db-4d28-9078-7d12e53a6c6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:01.068446 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a0e019-17db-4d28-9078-7d12e53a6c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:01.069253 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a0e019-17db-4d28-9078-7d12e53a6c6d admin admin] PATH_INFO: `/v3/projects/9e75e5e1ff1940ae8991a776a6c37699` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:01.071115 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a0e019-17db-4d28-9078-7d12e53a6c6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e75e5e1ff1940ae8991a776a6c37699)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:01.072710 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a0e019-17db-4d28-9078-7d12e53a6c6d admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:01.130057 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-35a0e019-17db-4d28-9078-7d12e53a6c6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e75e5e1ff1940ae8991a776a6c37699', 'request_id': 'req-35a0e019-17db-4d28-9078-7d12e53a6c6d'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:01.136409 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-35a0e019-17db-4d28-9078-7d12e53a6c6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e75e5e1ff1940ae8991a776a6c37699', 'request_id': 'req-35a0e019-17db-4d28-9078-7d12e53a6c6d'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:01.142904 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-35a0e019-17db-4d28-9078-7d12e53a6c6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e75e5e1ff1940ae8991a776a6c37699', 'request_id': 'req-35a0e019-17db-4d28-9078-7d12e53a6c6d'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:01.159716 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-35a0e019-17db-4d28-9078-7d12e53a6c6d admin admin] The token cache is being invalidate because project 9e75e5e1ff1940ae8991a776a6c37699 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:42:01.161616 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-35a0e019-17db-4d28-9078-7d12e53a6c6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:01.162383 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-35a0e019-17db-4d28-9078-7d12e53a6c6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:01.164164 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1054/2075] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:42:01 2022] DELETE /identity/v3/projects/9e75e5e1ff1940ae8991a776a6c37699 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:01.167972 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489a1bea-566c-45fb-ac83-add657f1d87c service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:01.177574 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489a1bea-566c-45fb-ac83-add657f1d87c service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:01.243471 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3801332-c3f0-40d8-bc1d-b5e782477344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:01.244344 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3801332-c3f0-40d8-bc1d-b5e782477344 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:01.244626 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3801332-c3f0-40d8-bc1d-b5e782477344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:01.244929 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3801332-c3f0-40d8-bc1d-b5e782477344 admin admin] PATH_INFO: `/v3/users/97334c4867e5485a98e19e0e98063b28` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:01.246215 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3801332-c3f0-40d8-bc1d-b5e782477344 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97334c4867e5485a98e19e0e98063b28)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:01.248538 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3801332-c3f0-40d8-bc1d-b5e782477344 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:01.295724 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c3801332-c3f0-40d8-bc1d-b5e782477344 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97334c4867e5485a98e19e0e98063b28'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:01.309022 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c3801332-c3f0-40d8-bc1d-b5e782477344 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97334c4867e5485a98e19e0e98063b28'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:01.309794 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c3801332-c3f0-40d8-bc1d-b5e782477344 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97334c4867e5485a98e19e0e98063b28'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:01.322418 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c3801332-c3f0-40d8-bc1d-b5e782477344 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97334c4867e5485a98e19e0e98063b28'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:01.323133 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c3801332-c3f0-40d8-bc1d-b5e782477344 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97334c4867e5485a98e19e0e98063b28'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:01.336841 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1022/2076] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:42:01 2022] DELETE /identity/v3/users/97334c4867e5485a98e19e0e98063b28 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:01.639193 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a0e019-17db-4d28-9078-7d12e53a6c6d admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:01.641684 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-35a0e019-17db-4d28-9078-7d12e53a6c6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:01.665204 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a0e019-17db-4d28-9078-7d12e53a6c6d admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:01.722532 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98de29b-c9ae-4a07-a467-4a70db4e67e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:01.723383 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98de29b-c9ae-4a07-a467-4a70db4e67e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:01.723660 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98de29b-c9ae-4a07-a467-4a70db4e67e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:01.723929 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98de29b-c9ae-4a07-a467-4a70db4e67e6 admin admin] PATH_INFO: `/v3/projects/dbb8d829098c47b0a3c46b010ac2fd17` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:01.725119 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98de29b-c9ae-4a07-a467-4a70db4e67e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbb8d829098c47b0a3c46b010ac2fd17)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:01.727076 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98de29b-c9ae-4a07-a467-4a70db4e67e6 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:01.773431 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-e98de29b-c9ae-4a07-a467-4a70db4e67e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbb8d829098c47b0a3c46b010ac2fd17', 'request_id': 'req-e98de29b-c9ae-4a07-a467-4a70db4e67e6'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:01.779773 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-e98de29b-c9ae-4a07-a467-4a70db4e67e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbb8d829098c47b0a3c46b010ac2fd17', 'request_id': 'req-e98de29b-c9ae-4a07-a467-4a70db4e67e6'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:01.780988 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:01.787593 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-e98de29b-c9ae-4a07-a467-4a70db4e67e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbb8d829098c47b0a3c46b010ac2fd17', 'request_id': 'req-e98de29b-c9ae-4a07-a467-4a70db4e67e6'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:01.801373 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-e98de29b-c9ae-4a07-a467-4a70db4e67e6 admin admin] The token cache is being invalidate because project dbb8d829098c47b0a3c46b010ac2fd17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:42:01.801666 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-e98de29b-c9ae-4a07-a467-4a70db4e67e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:01.802324 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-e98de29b-c9ae-4a07-a467-4a70db4e67e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:01.803858 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1055/2077] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:42:01 2022] DELETE /identity/v3/projects/dbb8d829098c47b0a3c46b010ac2fd17 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:01.808806 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b5d68d-b1ea-4d05-a218-45186162d1cf None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:01.809056 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b5d68d-b1ea-4d05-a218-45186162d1cf None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:01.809339 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b5d68d-b1ea-4d05-a218-45186162d1cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:01.827578 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-95b5d68d-b1ea-4d05-a218-45186162d1cf None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:01.828800 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-95b5d68d-b1ea-4d05-a218-45186162d1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:01.887990 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1023/2078] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:42:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:01.895925 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f5df89-9f8f-4cd1-9285-36bd1dc03b52 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:01.896217 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f5df89-9f8f-4cd1-9285-36bd1dc03b52 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:01.896498 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f5df89-9f8f-4cd1-9285-36bd1dc03b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:01.915195 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-90f5df89-9f8f-4cd1-9285-36bd1dc03b52 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:01.916410 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-90f5df89-9f8f-4cd1-9285-36bd1dc03b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:01.924380 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1056/2079] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:42:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:01.929832 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b5d68d-b1ea-4d05-a218-45186162d1cf None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:01.939773 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b5d68d-b1ea-4d05-a218-45186162d1cf None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:01.944402 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b061e7e-e263-4a60-9119-d61c019fb74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:01.945056 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b061e7e-e263-4a60-9119-d61c019fb74a admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:01.945338 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b061e7e-e263-4a60-9119-d61c019fb74a admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:01.945609 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b061e7e-e263-4a60-9119-d61c019fb74a admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:01.946359 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b061e7e-e263-4a60-9119-d61c019fb74a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:42:01.946664 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b061e7e-e263-4a60-9119-d61c019fb74a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:01.948262 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b061e7e-e263-4a60-9119-d61c019fb74a admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:01.959980 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1024/2080] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:42:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:42:01.964205 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f5df89-9f8f-4cd1-9285-36bd1dc03b52 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:01.974162 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f5df89-9f8f-4cd1-9285-36bd1dc03b52 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:01.978930 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9269435-c28f-4d41-af78-4943aa67eb58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:01.979577 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9269435-c28f-4d41-af78-4943aa67eb58 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:01.979864 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9269435-c28f-4d41-af78-4943aa67eb58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:01.980143 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9269435-c28f-4d41-af78-4943aa67eb58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:01.980826 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9269435-c28f-4d41-af78-4943aa67eb58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:01.982755 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9269435-c28f-4d41-af78-4943aa67eb58 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:02.008997 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1057/2081] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:42:01 2022] POST /identity/v3/projects => generated 391 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:02.013233 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b061e7e-e263-4a60-9119-d61c019fb74a admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:02.020075 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:02.024318 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b061e7e-e263-4a60-9119-d61c019fb74a admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:02.089465 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666d5036-8717-4d6b-b3ab-9f80a22d85ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:02.090248 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666d5036-8717-4d6b-b3ab-9f80a22d85ff admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:02.090539 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666d5036-8717-4d6b-b3ab-9f80a22d85ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:02.090823 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666d5036-8717-4d6b-b3ab-9f80a22d85ff admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:02.091519 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666d5036-8717-4d6b-b3ab-9f80a22d85ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:02.093466 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666d5036-8717-4d6b-b3ab-9f80a22d85ff admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:02.116833 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1025/2082] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:42:02 2022] POST /identity/v3/users => generated 296 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:02.121114 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9269435-c28f-4d41-af78-4943aa67eb58 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:02.132801 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9269435-c28f-4d41-af78-4943aa67eb58 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:02.138136 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc729d5c-51c6-4590-bd79-4dbfa36c4dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:02.138855 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc729d5c-51c6-4590-bd79-4dbfa36c4dde admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:02.139204 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc729d5c-51c6-4590-bd79-4dbfa36c4dde admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:02.139488 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc729d5c-51c6-4590-bd79-4dbfa36c4dde admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:02.140267 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc729d5c-51c6-4590-bd79-4dbfa36c4dde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:02.141921 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc729d5c-51c6-4590-bd79-4dbfa36c4dde admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:02.153765 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1058/2083] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:42:02.157756 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666d5036-8717-4d6b-b3ab-9f80a22d85ff admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:02.168919 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666d5036-8717-4d6b-b3ab-9f80a22d85ff admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:02.174016 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ef7634-1cda-4d7a-b15d-9395b057a69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:02.176143 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ef7634-1cda-4d7a-b15d-9395b057a69a admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:02.176434 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ef7634-1cda-4d7a-b15d-9395b057a69a admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:02.176713 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ef7634-1cda-4d7a-b15d-9395b057a69a admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:02.177451 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ef7634-1cda-4d7a-b15d-9395b057a69a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:02.179111 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ef7634-1cda-4d7a-b15d-9395b057a69a admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:02.193893 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1026/2084] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:02.195202 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc729d5c-51c6-4590-bd79-4dbfa36c4dde admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:02.205638 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc729d5c-51c6-4590-bd79-4dbfa36c4dde admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:02.211398 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f312fb86-4c05-4afe-b9ff-05c7801a378a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:02.212294 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f312fb86-4c05-4afe-b9ff-05c7801a378a admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:02.212582 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f312fb86-4c05-4afe-b9ff-05c7801a378a admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:02.212860 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f312fb86-4c05-4afe-b9ff-05c7801a378a admin admin] PATH_INFO: `/v3/projects/f135c27a5bd6425ab0ccf0fbe700f9fa/users/23ad9832bb144d4aa71a35be942a91e9/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:02.233044 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f312fb86-4c05-4afe-b9ff-05c7801a378a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f135c27a5bd6425ab0ccf0fbe700f9fa, user_id=23ad9832bb144d4aa71a35be942a91e9, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:02.235748 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f312fb86-4c05-4afe-b9ff-05c7801a378a admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:02.259578 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1059/2085] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:02 2022] PUT /identity/v3/projects/f135c27a5bd6425ab0ccf0fbe700f9fa/users/23ad9832bb144d4aa71a35be942a91e9/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:02.269978 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0264b4af-a671-4765-bd29-0a450191382e None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:02.270269 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0264b4af-a671-4765-bd29-0a450191382e None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:02.270545 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0264b4af-a671-4765-bd29-0a450191382e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:02.298681 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-0264b4af-a671-4765-bd29-0a450191382e None None] MFA Rules not processed for user `23ad9832bb144d4aa71a35be942a91e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:02.300411 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-0264b4af-a671-4765-bd29-0a450191382e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:02.369009 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1027/2086] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:42:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:02.373165 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f312fb86-4c05-4afe-b9ff-05c7801a378a admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:02.447696 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f312fb86-4c05-4afe-b9ff-05c7801a378a admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:02.452845 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb1061a-26d0-4fc4-b8c8-c1c8ca50e9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:02.453511 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb1061a-26d0-4fc4-b8c8-c1c8ca50e9fa admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:02.453825 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb1061a-26d0-4fc4-b8c8-c1c8ca50e9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:02.454100 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb1061a-26d0-4fc4-b8c8-c1c8ca50e9fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:02.454788 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb1061a-26d0-4fc4-b8c8-c1c8ca50e9fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:02.456927 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb1061a-26d0-4fc4-b8c8-c1c8ca50e9fa admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:02.483521 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1060/2087] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:42:02 2022] POST /identity/v3/projects => generated 393 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:02.488059 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0264b4af-a671-4765-bd29-0a450191382e None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:02.499253 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0264b4af-a671-4765-bd29-0a450191382e None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:02.560904 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f579f9d4-241a-4eb1-b23d-840727f5100b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:02.561584 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579f9d4-241a-4eb1-b23d-840727f5100b admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:02.561920 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579f9d4-241a-4eb1-b23d-840727f5100b admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:02.562202 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579f9d4-241a-4eb1-b23d-840727f5100b admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:02.562975 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f579f9d4-241a-4eb1-b23d-840727f5100b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:02.564883 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f579f9d4-241a-4eb1-b23d-840727f5100b admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:02.587859 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1028/2088] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:42:02 2022] POST /identity/v3/users => generated 303 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:02.589154 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb1061a-26d0-4fc4-b8c8-c1c8ca50e9fa admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:02.600672 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb1061a-26d0-4fc4-b8c8-c1c8ca50e9fa admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:02.605324 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d9c9a6-7bb5-4aec-ab37-68547f5c1306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:02.606030 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d9c9a6-7bb5-4aec-ab37-68547f5c1306 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:02.606319 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d9c9a6-7bb5-4aec-ab37-68547f5c1306 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:02.606597 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d9c9a6-7bb5-4aec-ab37-68547f5c1306 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:02.607331 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d9c9a6-7bb5-4aec-ab37-68547f5c1306 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:02.608931 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d9c9a6-7bb5-4aec-ab37-68547f5c1306 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:02.620457 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1061/2089] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:02.624884 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f579f9d4-241a-4eb1-b23d-840727f5100b admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:02.630175 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:02.636566 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f579f9d4-241a-4eb1-b23d-840727f5100b admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:02.641504 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c7216d-b949-49f9-aa71-60ff3be285db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:02.642414 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c7216d-b949-49f9-aa71-60ff3be285db admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:02.642709 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c7216d-b949-49f9-aa71-60ff3be285db admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:02.642990 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c7216d-b949-49f9-aa71-60ff3be285db admin admin] PATH_INFO: `/v3/projects/b219aa762bde4882948fc947f892c9c0/users/200f65dd54fa470abc89e4de73fa72fc/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:02.657826 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c7216d-b949-49f9-aa71-60ff3be285db admin admin] RBAC: Authorizing `identity:create_grant(project_id=b219aa762bde4882948fc947f892c9c0, user_id=200f65dd54fa470abc89e4de73fa72fc, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:02.660978 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c7216d-b949-49f9-aa71-60ff3be285db admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:02.676551 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1029/2090] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:02 2022] PUT /identity/v3/projects/b219aa762bde4882948fc947f892c9c0/users/200f65dd54fa470abc89e4de73fa72fc/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:02.680244 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d9c9a6-7bb5-4aec-ab37-68547f5c1306 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:02.690476 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d9c9a6-7bb5-4aec-ab37-68547f5c1306 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:02.741019 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:02.752160 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93990267-d7bf-46f2-bc21-803136ee94db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:02.753107 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93990267-d7bf-46f2-bc21-803136ee94db admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:02.753402 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93990267-d7bf-46f2-bc21-803136ee94db admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:02.753967 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93990267-d7bf-46f2-bc21-803136ee94db admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:02.754849 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93990267-d7bf-46f2-bc21-803136ee94db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:02.756746 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93990267-d7bf-46f2-bc21-803136ee94db admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:02.767736 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1062/2091] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:02 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:02.772176 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c7216d-b949-49f9-aa71-60ff3be285db admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:02.782858 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c7216d-b949-49f9-aa71-60ff3be285db admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:02.789416 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41667ccd-3463-49b1-af89-9f7149d0722e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:02.790363 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41667ccd-3463-49b1-af89-9f7149d0722e admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:02.790651 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41667ccd-3463-49b1-af89-9f7149d0722e admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:02.790939 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41667ccd-3463-49b1-af89-9f7149d0722e admin admin] PATH_INFO: `/v3/projects/b219aa762bde4882948fc947f892c9c0/users/200f65dd54fa470abc89e4de73fa72fc/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:02.792995 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41667ccd-3463-49b1-af89-9f7149d0722e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b219aa762bde4882948fc947f892c9c0, user_id=200f65dd54fa470abc89e4de73fa72fc, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:02.795822 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41667ccd-3463-49b1-af89-9f7149d0722e admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:02.808416 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1030/2092] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:02 2022] PUT /identity/v3/projects/b219aa762bde4882948fc947f892c9c0/users/200f65dd54fa470abc89e4de73fa72fc/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:02.821786 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911c4de-4735-43e2-bedc-585a9f565a75 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:02.822098 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911c4de-4735-43e2-bedc-585a9f565a75 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:02.822390 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911c4de-4735-43e2-bedc-585a9f565a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:02.851319 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-8911c4de-4735-43e2-bedc-585a9f565a75 None None] MFA Rules not processed for user `200f65dd54fa470abc89e4de73fa72fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:02.852558 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-8911c4de-4735-43e2-bedc-585a9f565a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:02.920298 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1063/2093] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:42:02 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:02.931153 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41667ccd-3463-49b1-af89-9f7149d0722e admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:02.932886 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-41667ccd-3463-49b1-af89-9f7149d0722e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:02.992123 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41667ccd-3463-49b1-af89-9f7149d0722e admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:02.996043 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef98752f-a7b8-49a6-bbe0-dd5fc72e5f1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:02.996770 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef98752f-a7b8-49a6-bbe0-dd5fc72e5f1a service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:02.997058 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef98752f-a7b8-49a6-bbe0-dd5fc72e5f1a service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:02.997335 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef98752f-a7b8-49a6-bbe0-dd5fc72e5f1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:03.007422 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef98752f-a7b8-49a6-bbe0-dd5fc72e5f1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:03.008857 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef98752f-a7b8-49a6-bbe0-dd5fc72e5f1a service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:03.021662 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1031/2094] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:02 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:42:03.079238 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8911c4de-4735-43e2-bedc-585a9f565a75 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:03.090835 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8911c4de-4735-43e2-bedc-585a9f565a75 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:03.094506 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fb9f2b-b135-48b6-96a9-657832421e4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:03.095157 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb9f2b-b135-48b6-96a9-657832421e4b service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:03.095442 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb9f2b-b135-48b6-96a9-657832421e4b service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:03.095720 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb9f2b-b135-48b6-96a9-657832421e4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:03.184897 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fb9f2b-b135-48b6-96a9-657832421e4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:03.186370 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fb9f2b-b135-48b6-96a9-657832421e4b service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:03.196229 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1064/2095] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:03 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:42:06.506122 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef98752f-a7b8-49a6-bbe0-dd5fc72e5f1a service cinder] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:06.507660 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-ef98752f-a7b8-49a6-bbe0-dd5fc72e5f1a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:06.573260 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef98752f-a7b8-49a6-bbe0-dd5fc72e5f1a service cinder] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:06.577933 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf437e2f-163c-4ed2-82f1-b36291c5a5c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:06.578801 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf437e2f-163c-4ed2-82f1-b36291c5a5c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:06.579088 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf437e2f-163c-4ed2-82f1-b36291c5a5c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:06.579366 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf437e2f-163c-4ed2-82f1-b36291c5a5c2 admin admin] PATH_INFO: `/v3/users/a790e4d2039942fdbbcd9eae9d99b854` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:06.580594 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf437e2f-163c-4ed2-82f1-b36291c5a5c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a790e4d2039942fdbbcd9eae9d99b854)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:06.583247 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf437e2f-163c-4ed2-82f1-b36291c5a5c2 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:06.636844 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-cf437e2f-163c-4ed2-82f1-b36291c5a5c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a790e4d2039942fdbbcd9eae9d99b854'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:06.651791 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-cf437e2f-163c-4ed2-82f1-b36291c5a5c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a790e4d2039942fdbbcd9eae9d99b854'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:06.652582 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-cf437e2f-163c-4ed2-82f1-b36291c5a5c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a790e4d2039942fdbbcd9eae9d99b854'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:06.657281 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:06.661186 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-cf437e2f-163c-4ed2-82f1-b36291c5a5c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a790e4d2039942fdbbcd9eae9d99b854'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:06.662581 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-cf437e2f-163c-4ed2-82f1-b36291c5a5c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a790e4d2039942fdbbcd9eae9d99b854'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:06.676970 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1032/2096] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:42:06 2022] DELETE /identity/v3/users/a790e4d2039942fdbbcd9eae9d99b854 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:06.681490 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519fad5f-34b4-4292-ac56-5077bf3860de None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:06.681996 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519fad5f-34b4-4292-ac56-5077bf3860de None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:06.682366 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519fad5f-34b4-4292-ac56-5077bf3860de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:06.700218 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-519fad5f-34b4-4292-ac56-5077bf3860de None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:06.701439 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-519fad5f-34b4-4292-ac56-5077bf3860de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:06.775737 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1065/2097] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:42:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:06.786559 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf437e2f-163c-4ed2-82f1-b36291c5a5c2 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:06.788005 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-cf437e2f-163c-4ed2-82f1-b36291c5a5c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:06.846223 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf437e2f-163c-4ed2-82f1-b36291c5a5c2 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:06.851413 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9003d2a0-22c1-48c8-b4e4-89a8689f3d7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:06.852082 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9003d2a0-22c1-48c8-b4e4-89a8689f3d7d service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:06.852368 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9003d2a0-22c1-48c8-b4e4-89a8689f3d7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:06.852789 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9003d2a0-22c1-48c8-b4e4-89a8689f3d7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:06.864767 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9003d2a0-22c1-48c8-b4e4-89a8689f3d7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:06.866256 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9003d2a0-22c1-48c8-b4e4-89a8689f3d7d service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:06.876264 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1033/2098] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:42:07.162018 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519fad5f-34b4-4292-ac56-5077bf3860de None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:07.163463 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-519fad5f-34b4-4292-ac56-5077bf3860de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:07.182247 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519fad5f-34b4-4292-ac56-5077bf3860de None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:07.187494 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57dbe62-75b6-4f52-9196-428a895310cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:07.188651 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57dbe62-75b6-4f52-9196-428a895310cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:07.189119 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57dbe62-75b6-4f52-9196-428a895310cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:07.189599 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57dbe62-75b6-4f52-9196-428a895310cf admin admin] PATH_INFO: `/v3/projects/b693f4e621bb4ffc848e598e7efa9559` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:07.191801 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57dbe62-75b6-4f52-9196-428a895310cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=b693f4e621bb4ffc848e598e7efa9559)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:07.195576 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57dbe62-75b6-4f52-9196-428a895310cf admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:07.208420 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:07.265174 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-f57dbe62-75b6-4f52-9196-428a895310cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b693f4e621bb4ffc848e598e7efa9559', 'request_id': 'req-f57dbe62-75b6-4f52-9196-428a895310cf'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:07.272270 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-f57dbe62-75b6-4f52-9196-428a895310cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b693f4e621bb4ffc848e598e7efa9559', 'request_id': 'req-f57dbe62-75b6-4f52-9196-428a895310cf'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:07.278444 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-f57dbe62-75b6-4f52-9196-428a895310cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b693f4e621bb4ffc848e598e7efa9559', 'request_id': 'req-f57dbe62-75b6-4f52-9196-428a895310cf'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:07.296413 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-f57dbe62-75b6-4f52-9196-428a895310cf admin admin] The token cache is being invalidate because project b693f4e621bb4ffc848e598e7efa9559 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:42:07.297009 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-f57dbe62-75b6-4f52-9196-428a895310cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:07.297741 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-f57dbe62-75b6-4f52-9196-428a895310cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:07.300707 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1066/2099] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:42:07 2022] DELETE /identity/v3/projects/b693f4e621bb4ffc848e598e7efa9559 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:07.595123 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9003d2a0-22c1-48c8-b4e4-89a8689f3d7d service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:07.597042 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-9003d2a0-22c1-48c8-b4e4-89a8689f3d7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:07.610057 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9003d2a0-22c1-48c8-b4e4-89a8689f3d7d service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:07.666456 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206390db-9d5c-49a8-be74-94da6c2a15f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:07.667247 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206390db-9d5c-49a8-be74-94da6c2a15f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:07.667535 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206390db-9d5c-49a8-be74-94da6c2a15f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:07.667814 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206390db-9d5c-49a8-be74-94da6c2a15f4 admin admin] PATH_INFO: `/v3/users/23ad9832bb144d4aa71a35be942a91e9` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:07.669010 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206390db-9d5c-49a8-be74-94da6c2a15f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23ad9832bb144d4aa71a35be942a91e9)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:07.670968 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206390db-9d5c-49a8-be74-94da6c2a15f4 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:07.717634 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-206390db-9d5c-49a8-be74-94da6c2a15f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23ad9832bb144d4aa71a35be942a91e9'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:07.730043 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-206390db-9d5c-49a8-be74-94da6c2a15f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23ad9832bb144d4aa71a35be942a91e9'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:07.730722 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-206390db-9d5c-49a8-be74-94da6c2a15f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23ad9832bb144d4aa71a35be942a91e9'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:07.738968 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-206390db-9d5c-49a8-be74-94da6c2a15f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23ad9832bb144d4aa71a35be942a91e9'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:07.740040 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-206390db-9d5c-49a8-be74-94da6c2a15f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23ad9832bb144d4aa71a35be942a91e9'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:07.754813 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1034/2100] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:42:07 2022] DELETE /identity/v3/users/23ad9832bb144d4aa71a35be942a91e9 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:07.759384 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf96eb19-3a2e-4cf2-b055-cb6d917a59d2 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:07.759855 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf96eb19-3a2e-4cf2-b055-cb6d917a59d2 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:07.759993 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf96eb19-3a2e-4cf2-b055-cb6d917a59d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:07.778272 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-cf96eb19-3a2e-4cf2-b055-cb6d917a59d2 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:07.779476 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-cf96eb19-3a2e-4cf2-b055-cb6d917a59d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:07.840387 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1067/2101] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:42:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:07.851039 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206390db-9d5c-49a8-be74-94da6c2a15f4 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:07.852468 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-206390db-9d5c-49a8-be74-94da6c2a15f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:07.863654 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:07.916728 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206390db-9d5c-49a8-be74-94da6c2a15f4 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:07.920765 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb503f81-3588-4227-b738-8bd262ee06a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:07.921421 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb503f81-3588-4227-b738-8bd262ee06a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:07.921736 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb503f81-3588-4227-b738-8bd262ee06a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:07.922080 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb503f81-3588-4227-b738-8bd262ee06a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:07.931637 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb503f81-3588-4227-b738-8bd262ee06a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:07.933122 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb503f81-3588-4227-b738-8bd262ee06a4 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:07.944259 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1035/2102] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:42:08.251836 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf96eb19-3a2e-4cf2-b055-cb6d917a59d2 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:08.253551 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-cf96eb19-3a2e-4cf2-b055-cb6d917a59d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:08.270554 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf96eb19-3a2e-4cf2-b055-cb6d917a59d2 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:08.281104 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5967abaa-3c81-4642-8bae-ebc439271b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:08.282050 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5967abaa-3c81-4642-8bae-ebc439271b9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:08.282424 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5967abaa-3c81-4642-8bae-ebc439271b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:08.282600 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5967abaa-3c81-4642-8bae-ebc439271b9a admin admin] PATH_INFO: `/v3/projects/f135c27a5bd6425ab0ccf0fbe700f9fa` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:08.283929 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5967abaa-3c81-4642-8bae-ebc439271b9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f135c27a5bd6425ab0ccf0fbe700f9fa)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:08.286069 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5967abaa-3c81-4642-8bae-ebc439271b9a admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:08.348592 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-5967abaa-3c81-4642-8bae-ebc439271b9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f135c27a5bd6425ab0ccf0fbe700f9fa', 'request_id': 'req-5967abaa-3c81-4642-8bae-ebc439271b9a'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:08.356727 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-5967abaa-3c81-4642-8bae-ebc439271b9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f135c27a5bd6425ab0ccf0fbe700f9fa', 'request_id': 'req-5967abaa-3c81-4642-8bae-ebc439271b9a'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:08.365292 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-5967abaa-3c81-4642-8bae-ebc439271b9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f135c27a5bd6425ab0ccf0fbe700f9fa', 'request_id': 'req-5967abaa-3c81-4642-8bae-ebc439271b9a'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:08.385148 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-5967abaa-3c81-4642-8bae-ebc439271b9a admin admin] The token cache is being invalidate because project f135c27a5bd6425ab0ccf0fbe700f9fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:42:08.385459 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-5967abaa-3c81-4642-8bae-ebc439271b9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:08.386191 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-5967abaa-3c81-4642-8bae-ebc439271b9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:08.387799 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1068/2103] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:42:08 2022] DELETE /identity/v3/projects/f135c27a5bd6425ab0ccf0fbe700f9fa => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:08.391866 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb503f81-3588-4227-b738-8bd262ee06a4 service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:08.402368 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb503f81-3588-4227-b738-8bd262ee06a4 service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:08.473393 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741b7141-da7e-43aa-8fda-f68bddce395d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:08.474513 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b7141-da7e-43aa-8fda-f68bddce395d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:08.477993 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b7141-da7e-43aa-8fda-f68bddce395d admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:08.478466 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b7141-da7e-43aa-8fda-f68bddce395d admin admin] PATH_INFO: `/v3/users/200f65dd54fa470abc89e4de73fa72fc` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:08.486293 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741b7141-da7e-43aa-8fda-f68bddce395d admin admin] RBAC: Authorizing `identity:delete_user(user_id=200f65dd54fa470abc89e4de73fa72fc)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:08.488926 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741b7141-da7e-43aa-8fda-f68bddce395d admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:08.550561 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-741b7141-da7e-43aa-8fda-f68bddce395d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '200f65dd54fa470abc89e4de73fa72fc'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:08.567547 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-741b7141-da7e-43aa-8fda-f68bddce395d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '200f65dd54fa470abc89e4de73fa72fc'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:08.568275 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-741b7141-da7e-43aa-8fda-f68bddce395d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '200f65dd54fa470abc89e4de73fa72fc'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:08.577547 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-741b7141-da7e-43aa-8fda-f68bddce395d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '200f65dd54fa470abc89e4de73fa72fc'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:08.578574 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-741b7141-da7e-43aa-8fda-f68bddce395d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '200f65dd54fa470abc89e4de73fa72fc'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:08.600314 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1036/2104] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:42:08 2022] DELETE /identity/v3/users/200f65dd54fa470abc89e4de73fa72fc => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:09.072470 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5967abaa-3c81-4642-8bae-ebc439271b9a admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:09.074051 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-5967abaa-3c81-4642-8bae-ebc439271b9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:09.088744 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5967abaa-3c81-4642-8bae-ebc439271b9a admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:09.116888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:09.163341 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad22a62-2679-4a8b-b828-85dd236fde8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:09.164211 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad22a62-2679-4a8b-b828-85dd236fde8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:09.164492 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad22a62-2679-4a8b-b828-85dd236fde8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:09.164767 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad22a62-2679-4a8b-b828-85dd236fde8b admin admin] PATH_INFO: `/v3/projects/b219aa762bde4882948fc947f892c9c0` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:09.165983 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad22a62-2679-4a8b-b828-85dd236fde8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b219aa762bde4882948fc947f892c9c0)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:09.167919 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad22a62-2679-4a8b-b828-85dd236fde8b admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:09.227013 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7ad22a62-2679-4a8b-b828-85dd236fde8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b219aa762bde4882948fc947f892c9c0', 'request_id': 'req-7ad22a62-2679-4a8b-b828-85dd236fde8b'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:09.233561 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7ad22a62-2679-4a8b-b828-85dd236fde8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b219aa762bde4882948fc947f892c9c0', 'request_id': 'req-7ad22a62-2679-4a8b-b828-85dd236fde8b'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:09.239940 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7ad22a62-2679-4a8b-b828-85dd236fde8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b219aa762bde4882948fc947f892c9c0', 'request_id': 'req-7ad22a62-2679-4a8b-b828-85dd236fde8b'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:09.254370 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7ad22a62-2679-4a8b-b828-85dd236fde8b admin admin] The token cache is being invalidate because project b219aa762bde4882948fc947f892c9c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:42:09.254656 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7ad22a62-2679-4a8b-b828-85dd236fde8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:09.255675 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7ad22a62-2679-4a8b-b828-85dd236fde8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:09.257466 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1069/2105] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:42:09 2022] DELETE /identity/v3/projects/b219aa762bde4882948fc947f892c9c0 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:09.262411 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe3511-6c20-4ca6-8cfb-d264c5214865 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:09.262784 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe3511-6c20-4ca6-8cfb-d264c5214865 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:09.263229 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe3511-6c20-4ca6-8cfb-d264c5214865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:09.283397 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-57fe3511-6c20-4ca6-8cfb-d264c5214865 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:09.285270 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-57fe3511-6c20-4ca6-8cfb-d264c5214865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:09.314287 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:09.353743 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1037/2106] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:42:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:09.359978 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c25e280-a6f2-4c11-8edd-d894fa1498ad None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:09.360267 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c25e280-a6f2-4c11-8edd-d894fa1498ad None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:09.360545 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c25e280-a6f2-4c11-8edd-d894fa1498ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:09.386470 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-0c25e280-a6f2-4c11-8edd-d894fa1498ad None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:09.387719 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-0c25e280-a6f2-4c11-8edd-d894fa1498ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:09.395745 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1070/2107] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:42:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:09.400042 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe3511-6c20-4ca6-8cfb-d264c5214865 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:09.411070 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe3511-6c20-4ca6-8cfb-d264c5214865 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:09.422473 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a6f1e-8136-44e2-9ac3-5895a0b76d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:09.423231 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a6f1e-8136-44e2-9ac3-5895a0b76d6e admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:09.423605 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a6f1e-8136-44e2-9ac3-5895a0b76d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:09.423939 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a6f1e-8136-44e2-9ac3-5895a0b76d6e admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:09.427181 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272a6f1e-8136-44e2-9ac3-5895a0b76d6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:42:09.427924 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272a6f1e-8136-44e2-9ac3-5895a0b76d6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:09.430928 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272a6f1e-8136-44e2-9ac3-5895a0b76d6e admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:09.451393 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1038/2108] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:42:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:42:09.455748 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c25e280-a6f2-4c11-8edd-d894fa1498ad None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:09.465404 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c25e280-a6f2-4c11-8edd-d894fa1498ad None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:09.470320 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46afd8-ecba-4e33-b10e-2dcd02eb6121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:09.471813 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46afd8-ecba-4e33-b10e-2dcd02eb6121 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:09.472575 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46afd8-ecba-4e33-b10e-2dcd02eb6121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:09.472575 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46afd8-ecba-4e33-b10e-2dcd02eb6121 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:09.473498 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f46afd8-ecba-4e33-b10e-2dcd02eb6121 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:09.475522 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f46afd8-ecba-4e33-b10e-2dcd02eb6121 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:09.513902 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1071/2109] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:42:09 2022] POST /identity/v3/projects => generated 377 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:09.516778 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a6f1e-8136-44e2-9ac3-5895a0b76d6e admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:09.528101 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a6f1e-8136-44e2-9ac3-5895a0b76d6e admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:09.599598 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ac302a-ef5c-4ca9-97b7-41583efe7862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:09.600477 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac302a-ef5c-4ca9-97b7-41583efe7862 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:09.601018 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac302a-ef5c-4ca9-97b7-41583efe7862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:09.601136 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac302a-ef5c-4ca9-97b7-41583efe7862 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:09.601811 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ac302a-ef5c-4ca9-97b7-41583efe7862 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:09.603903 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ac302a-ef5c-4ca9-97b7-41583efe7862 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:09.626683 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1039/2110] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:42:09 2022] POST /identity/v3/users => generated 289 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:09.630770 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46afd8-ecba-4e33-b10e-2dcd02eb6121 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:09.644890 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46afd8-ecba-4e33-b10e-2dcd02eb6121 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:09.650750 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701a2cfd-8c86-41f5-8642-66b7e2dd3cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:09.651582 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a2cfd-8c86-41f5-8642-66b7e2dd3cd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:09.651937 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a2cfd-8c86-41f5-8642-66b7e2dd3cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:09.652331 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a2cfd-8c86-41f5-8642-66b7e2dd3cd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:09.653068 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701a2cfd-8c86-41f5-8642-66b7e2dd3cd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:09.654921 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701a2cfd-8c86-41f5-8642-66b7e2dd3cd6 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:09.666982 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1072/2111] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:09.670989 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ac302a-ef5c-4ca9-97b7-41583efe7862 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:09.681582 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ac302a-ef5c-4ca9-97b7-41583efe7862 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:09.686427 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6714825-72bb-48ae-90e0-e3a4cb0e5fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:09.687102 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6714825-72bb-48ae-90e0-e3a4cb0e5fda admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:09.687390 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6714825-72bb-48ae-90e0-e3a4cb0e5fda admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:09.687672 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6714825-72bb-48ae-90e0-e3a4cb0e5fda admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:09.688399 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6714825-72bb-48ae-90e0-e3a4cb0e5fda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:09.690231 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6714825-72bb-48ae-90e0-e3a4cb0e5fda admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:09.701187 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1040/2112] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:09.706269 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701a2cfd-8c86-41f5-8642-66b7e2dd3cd6 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:09.718679 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701a2cfd-8c86-41f5-8642-66b7e2dd3cd6 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:09.724629 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae5f2b5-56b9-4af7-b853-c9586081e59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:09.725697 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5f2b5-56b9-4af7-b853-c9586081e59e admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:09.726096 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5f2b5-56b9-4af7-b853-c9586081e59e admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:09.726464 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5f2b5-56b9-4af7-b853-c9586081e59e admin admin] PATH_INFO: `/v3/projects/001f8e102921445ba9cb4bbd17777a06/users/c27f59913abc4b8fa7b9dd9a50f6f4a4/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:09.743042 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae5f2b5-56b9-4af7-b853-c9586081e59e admin admin] RBAC: Authorizing `identity:create_grant(project_id=001f8e102921445ba9cb4bbd17777a06, user_id=c27f59913abc4b8fa7b9dd9a50f6f4a4, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:09.745923 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae5f2b5-56b9-4af7-b853-c9586081e59e admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:09.761319 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1073/2113] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:09 2022] PUT /identity/v3/projects/001f8e102921445ba9cb4bbd17777a06/users/c27f59913abc4b8fa7b9dd9a50f6f4a4/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:09.772834 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d044716f-d5b4-4f45-8177-4e27db8984f9 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:09.773130 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d044716f-d5b4-4f45-8177-4e27db8984f9 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:09.773415 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d044716f-d5b4-4f45-8177-4e27db8984f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:09.803172 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-d044716f-d5b4-4f45-8177-4e27db8984f9 None None] MFA Rules not processed for user `c27f59913abc4b8fa7b9dd9a50f6f4a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:09.804525 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d044716f-d5b4-4f45-8177-4e27db8984f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:09.883371 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1041/2114] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:42:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:42:09.890062 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d044716f-d5b4-4f45-8177-4e27db8984f9 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:09.890197 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae5f2b5-56b9-4af7-b853-c9586081e59e admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:09.891399 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d044716f-d5b4-4f45-8177-4e27db8984f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:09.922775 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:09.942450 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:09.961245 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae5f2b5-56b9-4af7-b853-c9586081e59e admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:09.962010 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d044716f-d5b4-4f45-8177-4e27db8984f9 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:09.965930 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af2d808-b2cf-4ef2-9166-2be67b950a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:09.966596 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af2d808-b2cf-4ef2-9166-2be67b950a14 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:09.966816 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c49dcb-9b22-4bf9-9cb1-276b5d30665c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:09.966949 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af2d808-b2cf-4ef2-9166-2be67b950a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:09.967153 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af2d808-b2cf-4ef2-9166-2be67b950a14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:09.967683 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c49dcb-9b22-4bf9-9cb1-276b5d30665c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:09.967847 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af2d808-b2cf-4ef2-9166-2be67b950a14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:09.967970 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c49dcb-9b22-4bf9-9cb1-276b5d30665c admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:09.968247 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c49dcb-9b22-4bf9-9cb1-276b5d30665c admin admin] PATH_INFO: `/v3/users/84a3046facfb40ec893eed6aae236a0c` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:09.969445 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c49dcb-9b22-4bf9-9cb1-276b5d30665c admin admin] RBAC: Authorizing `identity:delete_user(user_id=84a3046facfb40ec893eed6aae236a0c)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:09.969783 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af2d808-b2cf-4ef2-9166-2be67b950a14 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:09.972056 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c49dcb-9b22-4bf9-9cb1-276b5d30665c admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:09.997124 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1074/2115] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:42:09 2022] POST /identity/v3/projects => generated 375 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:10.001567 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af2d808-b2cf-4ef2-9166-2be67b950a14 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:10.023730 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-93c49dcb-9b22-4bf9-9cb1-276b5d30665c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84a3046facfb40ec893eed6aae236a0c'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:10.035544 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-93c49dcb-9b22-4bf9-9cb1-276b5d30665c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84a3046facfb40ec893eed6aae236a0c'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:10.036243 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-93c49dcb-9b22-4bf9-9cb1-276b5d30665c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84a3046facfb40ec893eed6aae236a0c'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:10.043892 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-93c49dcb-9b22-4bf9-9cb1-276b5d30665c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84a3046facfb40ec893eed6aae236a0c'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:10.044888 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-93c49dcb-9b22-4bf9-9cb1-276b5d30665c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84a3046facfb40ec893eed6aae236a0c'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:10.061150 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1042/2116] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:42:09 2022] DELETE /identity/v3/users/84a3046facfb40ec893eed6aae236a0c => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:10.066170 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d9cb9-32d8-4561-9feb-24e26908a65c None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:10.066475 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d9cb9-32d8-4561-9feb-24e26908a65c None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:10.066826 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d9cb9-32d8-4561-9feb-24e26908a65c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:10.073989 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af2d808-b2cf-4ef2-9166-2be67b950a14 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:10.088221 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-5a6d9cb9-32d8-4561-9feb-24e26908a65c None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:10.089417 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-5a6d9cb9-32d8-4561-9feb-24e26908a65c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:10.135696 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a01fed1-db9d-407d-bdad-f313a46e0dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:10.136374 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a01fed1-db9d-407d-bdad-f313a46e0dd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:10.136660 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a01fed1-db9d-407d-bdad-f313a46e0dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:10.136936 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a01fed1-db9d-407d-bdad-f313a46e0dd9 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:10.137625 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a01fed1-db9d-407d-bdad-f313a46e0dd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:10.139578 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a01fed1-db9d-407d-bdad-f313a46e0dd9 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:10.153898 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1043/2117] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:42:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:42:10.155192 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17c3990-7aea-469d-9733-dddd5d75cdb1 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:10.155478 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17c3990-7aea-469d-9733-dddd5d75cdb1 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:10.155751 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17c3990-7aea-469d-9733-dddd5d75cdb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:10.168108 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1075/2118] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:42:10 2022] POST /identity/v3/users => generated 288 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:42:10.168108 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a01fed1-db9d-407d-bdad-f313a46e0dd9 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:10.169790 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-1a01fed1-db9d-407d-bdad-f313a46e0dd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:10.174714 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-c17c3990-7aea-469d-9733-dddd5d75cdb1 None None] MFA Rules not processed for user `740d843e5abb439b954996aaf841ed63`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:10.175824 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-c17c3990-7aea-469d-9733-dddd5d75cdb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:10.222174 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:10.238880 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1044/2119] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:42:10 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:10.239812 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17c3990-7aea-469d-9733-dddd5d75cdb1 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:10.241142 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-c17c3990-7aea-469d-9733-dddd5d75cdb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:10.250474 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a01fed1-db9d-407d-bdad-f313a46e0dd9 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:10.256379 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aa9253-2e39-4a55-8ab9-1d9a17a689ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:10.257047 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa9253-2e39-4a55-8ab9-1d9a17a689ae service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:10.257262 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17c3990-7aea-469d-9733-dddd5d75cdb1 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:10.257365 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa9253-2e39-4a55-8ab9-1d9a17a689ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:10.257600 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa9253-2e39-4a55-8ab9-1d9a17a689ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:10.262449 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192b178f-0077-4dcd-8dbc-c51bfdb5e385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:10.263149 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b178f-0077-4dcd-8dbc-c51bfdb5e385 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:10.263435 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b178f-0077-4dcd-8dbc-c51bfdb5e385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:10.263709 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b178f-0077-4dcd-8dbc-c51bfdb5e385 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:10.264443 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192b178f-0077-4dcd-8dbc-c51bfdb5e385 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:10.266104 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192b178f-0077-4dcd-8dbc-c51bfdb5e385 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:10.268845 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aa9253-2e39-4a55-8ab9-1d9a17a689ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:10.270348 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aa9253-2e39-4a55-8ab9-1d9a17a689ae service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:10.277804 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1045/2120] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:10 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:10.280655 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1076/2121] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:42:10.281709 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192b178f-0077-4dcd-8dbc-c51bfdb5e385 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:10.293833 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192b178f-0077-4dcd-8dbc-c51bfdb5e385 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:10.299830 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed884817-8b4a-485a-93cd-a5d762952312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:10.300579 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed884817-8b4a-485a-93cd-a5d762952312 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:10.300939 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed884817-8b4a-485a-93cd-a5d762952312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:10.301288 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed884817-8b4a-485a-93cd-a5d762952312 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:10.302127 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed884817-8b4a-485a-93cd-a5d762952312 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:10.304201 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed884817-8b4a-485a-93cd-a5d762952312 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:10.315969 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1046/2122] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:10.320245 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aa9253-2e39-4a55-8ab9-1d9a17a689ae service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:10.330197 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aa9253-2e39-4a55-8ab9-1d9a17a689ae service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:10.335881 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf9df1e-0533-4d54-95ac-5863da11b2b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:10.336960 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf9df1e-0533-4d54-95ac-5863da11b2b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:10.337432 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf9df1e-0533-4d54-95ac-5863da11b2b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:10.337762 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf9df1e-0533-4d54-95ac-5863da11b2b3 admin admin] PATH_INFO: `/v3/projects/c30869954ce048cd86f483c8a1735750/users/d2219a6b9ad44796912db59b0bffad9e/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:10.347022 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:10.359880 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf9df1e-0533-4d54-95ac-5863da11b2b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c30869954ce048cd86f483c8a1735750, user_id=d2219a6b9ad44796912db59b0bffad9e, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:10.363838 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf9df1e-0533-4d54-95ac-5863da11b2b3 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:10.387750 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1077/2123] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:10 2022] PUT /identity/v3/projects/c30869954ce048cd86f483c8a1735750/users/d2219a6b9ad44796912db59b0bffad9e/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:10.399359 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb99881-339c-4711-9538-f04f1eb88af0 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:10.399662 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb99881-339c-4711-9538-f04f1eb88af0 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:10.399945 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb99881-339c-4711-9538-f04f1eb88af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:10.428710 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-8cb99881-339c-4711-9538-f04f1eb88af0 None None] MFA Rules not processed for user `d2219a6b9ad44796912db59b0bffad9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:10.430128 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-8cb99881-339c-4711-9538-f04f1eb88af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:10.503173 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1047/2124] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:42:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:10.507588 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf9df1e-0533-4d54-95ac-5863da11b2b3 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:10.595348 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf9df1e-0533-4d54-95ac-5863da11b2b3 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:10.600802 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c47a9a-d531-4c28-8ac1-97da1932875a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:10.601430 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c47a9a-d531-4c28-8ac1-97da1932875a admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:10.601743 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c47a9a-d531-4c28-8ac1-97da1932875a admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:10.602029 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c47a9a-d531-4c28-8ac1-97da1932875a admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:10.602764 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c47a9a-d531-4c28-8ac1-97da1932875a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:10.605125 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c47a9a-d531-4c28-8ac1-97da1932875a admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:10.645735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1078/2125] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:42:10 2022] POST /identity/v3/projects => generated 375 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:10.649292 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb99881-339c-4711-9538-f04f1eb88af0 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:10.661020 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb99881-339c-4711-9538-f04f1eb88af0 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:10.747405 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d908ccc5-4565-4ee1-abb9-ca36839fcf29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:10.748603 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908ccc5-4565-4ee1-abb9-ca36839fcf29 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:10.749025 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908ccc5-4565-4ee1-abb9-ca36839fcf29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:10.749444 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908ccc5-4565-4ee1-abb9-ca36839fcf29 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:10.750315 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d908ccc5-4565-4ee1-abb9-ca36839fcf29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:10.753038 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d908ccc5-4565-4ee1-abb9-ca36839fcf29 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:10.784403 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1048/2126] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:42:10 2022] POST /identity/v3/users => generated 294 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:10.788233 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c47a9a-d531-4c28-8ac1-97da1932875a admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:10.799587 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c47a9a-d531-4c28-8ac1-97da1932875a admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:10.804417 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a77c69a-8e48-454a-b143-b6a3dc20fde9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:10.805104 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a77c69a-8e48-454a-b143-b6a3dc20fde9 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:10.805469 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a77c69a-8e48-454a-b143-b6a3dc20fde9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:10.805924 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a77c69a-8e48-454a-b143-b6a3dc20fde9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:10.806675 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a77c69a-8e48-454a-b143-b6a3dc20fde9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:10.808605 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a77c69a-8e48-454a-b143-b6a3dc20fde9 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:10.821531 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1079/2127] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:10.826002 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d908ccc5-4565-4ee1-abb9-ca36839fcf29 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:10.840471 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d908ccc5-4565-4ee1-abb9-ca36839fcf29 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:10.848838 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a04f90e-de8b-4de5-85be-a1eb47db5692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:10.850070 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04f90e-de8b-4de5-85be-a1eb47db5692 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:10.850682 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04f90e-de8b-4de5-85be-a1eb47db5692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:10.851143 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04f90e-de8b-4de5-85be-a1eb47db5692 admin admin] PATH_INFO: `/v3/projects/a2ce5f44201e484ebb88033bf3f8c65f/users/5fca590160504ca88d67ee507616f126/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:10.870258 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a04f90e-de8b-4de5-85be-a1eb47db5692 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2ce5f44201e484ebb88033bf3f8c65f, user_id=5fca590160504ca88d67ee507616f126, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:10.873308 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a04f90e-de8b-4de5-85be-a1eb47db5692 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:10.889823 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1049/2128] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:10 2022] PUT /identity/v3/projects/a2ce5f44201e484ebb88033bf3f8c65f/users/5fca590160504ca88d67ee507616f126/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:10.894332 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a77c69a-8e48-454a-b143-b6a3dc20fde9 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:10.908417 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a77c69a-8e48-454a-b143-b6a3dc20fde9 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:10.989303 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc5a599-7a86-42c8-8b9c-f7693166df81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:10.989986 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5a599-7a86-42c8-8b9c-f7693166df81 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:10.990270 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5a599-7a86-42c8-8b9c-f7693166df81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:10.990545 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5a599-7a86-42c8-8b9c-f7693166df81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:10.991287 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc5a599-7a86-42c8-8b9c-f7693166df81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:10.993166 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc5a599-7a86-42c8-8b9c-f7693166df81 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:11.010529 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1080/2129] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:10 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:11.012354 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a04f90e-de8b-4de5-85be-a1eb47db5692 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:11.024883 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a04f90e-de8b-4de5-85be-a1eb47db5692 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:11.029647 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed84b06a-d38b-4efb-9213-d6fbc8c69670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:11.030580 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed84b06a-d38b-4efb-9213-d6fbc8c69670 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:11.030863 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed84b06a-d38b-4efb-9213-d6fbc8c69670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:11.031137 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed84b06a-d38b-4efb-9213-d6fbc8c69670 admin admin] PATH_INFO: `/v3/projects/a2ce5f44201e484ebb88033bf3f8c65f/users/5fca590160504ca88d67ee507616f126/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:11.033155 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed84b06a-d38b-4efb-9213-d6fbc8c69670 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2ce5f44201e484ebb88033bf3f8c65f, user_id=5fca590160504ca88d67ee507616f126, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:11.036627 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed84b06a-d38b-4efb-9213-d6fbc8c69670 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:11.049132 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1050/2130] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:11 2022] PUT /identity/v3/projects/a2ce5f44201e484ebb88033bf3f8c65f/users/5fca590160504ca88d67ee507616f126/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:11.059334 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07549be7-9696-4a07-9cab-a05d04ded973 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:11.059625 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07549be7-9696-4a07-9cab-a05d04ded973 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:11.059899 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07549be7-9696-4a07-9cab-a05d04ded973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:11.089896 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-07549be7-9696-4a07-9cab-a05d04ded973 None None] MFA Rules not processed for user `5fca590160504ca88d67ee507616f126`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:11.090571 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-07549be7-9696-4a07-9cab-a05d04ded973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:11.150193 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:11.179474 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed84b06a-d38b-4efb-9213-d6fbc8c69670 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:11.185909 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1081/2131] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:42:11 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:11.186792 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-ed84b06a-d38b-4efb-9213-d6fbc8c69670 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:11.196653 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07549be7-9696-4a07-9cab-a05d04ded973 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:11.202662 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-07549be7-9696-4a07-9cab-a05d04ded973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:11.226460 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:11.309541 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07549be7-9696-4a07-9cab-a05d04ded973 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:11.319900 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed84b06a-d38b-4efb-9213-d6fbc8c69670 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:11.324905 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53907610-d7d9-43ad-9f42-333a0cb68fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:11.325057 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d52b035-c928-48e8-afb1-d7d307a5f007 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:11.325667 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52b035-c928-48e8-afb1-d7d307a5f007 service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:11.325807 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53907610-d7d9-43ad-9f42-333a0cb68fe3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:11.326011 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52b035-c928-48e8-afb1-d7d307a5f007 service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:11.326110 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53907610-d7d9-43ad-9f42-333a0cb68fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:11.326268 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52b035-c928-48e8-afb1-d7d307a5f007 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:11.326376 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53907610-d7d9-43ad-9f42-333a0cb68fe3 admin admin] PATH_INFO: `/v3/projects/7d1284e4a0944fee92ffc37d2120306a` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:11.327603 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53907610-d7d9-43ad-9f42-333a0cb68fe3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d1284e4a0944fee92ffc37d2120306a)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:11.328486 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-1d52b035-c928-48e8-afb1-d7d307a5f007 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:11.329529 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53907610-d7d9-43ad-9f42-333a0cb68fe3 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:11.451217 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-53907610-d7d9-43ad-9f42-333a0cb68fe3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d1284e4a0944fee92ffc37d2120306a', 'request_id': 'req-53907610-d7d9-43ad-9f42-333a0cb68fe3'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:11.460462 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-53907610-d7d9-43ad-9f42-333a0cb68fe3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d1284e4a0944fee92ffc37d2120306a', 'request_id': 'req-53907610-d7d9-43ad-9f42-333a0cb68fe3'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:11.466243 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d52b035-c928-48e8-afb1-d7d307a5f007 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:11.467379 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-53907610-d7d9-43ad-9f42-333a0cb68fe3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d1284e4a0944fee92ffc37d2120306a', 'request_id': 'req-53907610-d7d9-43ad-9f42-333a0cb68fe3'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:11.475200 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d52b035-c928-48e8-afb1-d7d307a5f007 service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:11.487963 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-53907610-d7d9-43ad-9f42-333a0cb68fe3 admin admin] The token cache is being invalidate because project 7d1284e4a0944fee92ffc37d2120306a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:42:11.488496 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-53907610-d7d9-43ad-9f42-333a0cb68fe3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:11.488909 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1082/2132] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:42:11.489257 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-53907610-d7d9-43ad-9f42-333a0cb68fe3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:11.491293 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1051/2133] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:42:11 2022] DELETE /identity/v3/projects/7d1284e4a0944fee92ffc37d2120306a => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:42:11.505367 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a1c5f1-0ffd-40fb-a874-af7b9a528e03 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:11.505660 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a1c5f1-0ffd-40fb-a874-af7b9a528e03 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:11.508953 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a1c5f1-0ffd-40fb-a874-af7b9a528e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:11.530343 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-80a1c5f1-0ffd-40fb-a874-af7b9a528e03 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:11.531718 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-80a1c5f1-0ffd-40fb-a874-af7b9a528e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:11.623378 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1083/2134] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:42:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:11.631693 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f907f6-e754-47b5-80a6-49440702b12f None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:11.632026 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f907f6-e754-47b5-80a6-49440702b12f None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:11.632294 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f907f6-e754-47b5-80a6-49440702b12f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:11.661229 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-d5f907f6-e754-47b5-80a6-49440702b12f None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:11.662519 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d5f907f6-e754-47b5-80a6-49440702b12f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:11.676289 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1052/2135] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:42:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:11.677870 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a1c5f1-0ffd-40fb-a874-af7b9a528e03 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:11.696631 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a1c5f1-0ffd-40fb-a874-af7b9a528e03 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:11.703244 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb9271-0d41-44ab-bfbb-9852393afa2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:11.703925 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb9271-0d41-44ab-bfbb-9852393afa2c admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:11.704210 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb9271-0d41-44ab-bfbb-9852393afa2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:11.704483 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb9271-0d41-44ab-bfbb-9852393afa2c admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:11.705608 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcb9271-0d41-44ab-bfbb-9852393afa2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:42:11.705608 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcb9271-0d41-44ab-bfbb-9852393afa2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:11.707371 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcb9271-0d41-44ab-bfbb-9852393afa2c admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:11.731593 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1084/2136] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:42:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:42:11.733019 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f907f6-e754-47b5-80a6-49440702b12f None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:11.749520 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f907f6-e754-47b5-80a6-49440702b12f None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:11.756405 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b956f-1e92-48b4-9aec-7884b0d44ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:11.758761 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b956f-1e92-48b4-9aec-7884b0d44ccc admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:11.759239 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b956f-1e92-48b4-9aec-7884b0d44ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:11.759547 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b956f-1e92-48b4-9aec-7884b0d44ccc admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:11.760268 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1b956f-1e92-48b4-9aec-7884b0d44ccc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:11.762597 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1b956f-1e92-48b4-9aec-7884b0d44ccc admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:11.802292 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1053/2137] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:42:11 2022] POST /identity/v3/projects => generated 365 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:11.803170 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb9271-0d41-44ab-bfbb-9852393afa2c admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:11.815535 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb9271-0d41-44ab-bfbb-9852393afa2c admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:11.840754 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:11.900242 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8de7a6-d999-4ee6-a68b-65a8ce108a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:11.901252 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8de7a6-d999-4ee6-a68b-65a8ce108a91 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:11.901604 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8de7a6-d999-4ee6-a68b-65a8ce108a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:11.902007 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8de7a6-d999-4ee6-a68b-65a8ce108a91 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:11.902906 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8de7a6-d999-4ee6-a68b-65a8ce108a91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:11.905167 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8de7a6-d999-4ee6-a68b-65a8ce108a91 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:11.941196 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1085/2138] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:42:11 2022] POST /identity/v3/users => generated 283 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:42:11.942533 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b956f-1e92-48b4-9aec-7884b0d44ccc admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:11.958695 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b956f-1e92-48b4-9aec-7884b0d44ccc admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:11.967067 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7509a69-5c7e-46e1-922f-e44676219f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:11.968039 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7509a69-5c7e-46e1-922f-e44676219f60 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:11.968667 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7509a69-5c7e-46e1-922f-e44676219f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:11.969150 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7509a69-5c7e-46e1-922f-e44676219f60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:11.970193 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7509a69-5c7e-46e1-922f-e44676219f60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:11.973151 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7509a69-5c7e-46e1-922f-e44676219f60 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:11.990079 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1054/2139] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:11 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:11.991261 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8de7a6-d999-4ee6-a68b-65a8ce108a91 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:12.005628 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8de7a6-d999-4ee6-a68b-65a8ce108a91 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:12.011838 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af923cc1-8416-4f15-bde8-a26df7c60954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:12.012651 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af923cc1-8416-4f15-bde8-a26df7c60954 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:12.013032 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af923cc1-8416-4f15-bde8-a26df7c60954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:12.013387 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af923cc1-8416-4f15-bde8-a26df7c60954 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:12.014247 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af923cc1-8416-4f15-bde8-a26df7c60954 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:12.016111 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af923cc1-8416-4f15-bde8-a26df7c60954 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:12.027888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1086/2140] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:12.032129 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7509a69-5c7e-46e1-922f-e44676219f60 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:12.043054 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7509a69-5c7e-46e1-922f-e44676219f60 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:12.047904 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61d07b6-e95e-4e6f-871d-e074c1510a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:12.048859 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d07b6-e95e-4e6f-871d-e074c1510a88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:12.049144 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d07b6-e95e-4e6f-871d-e074c1510a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:12.049477 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d07b6-e95e-4e6f-871d-e074c1510a88 admin admin] PATH_INFO: `/v3/projects/19733ce9e7f14e8da367fa7a04dfdc74/users/7fd0bf40927d4d86866e0763842a23b3/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:12.065700 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61d07b6-e95e-4e6f-871d-e074c1510a88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19733ce9e7f14e8da367fa7a04dfdc74, user_id=7fd0bf40927d4d86866e0763842a23b3, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:12.068453 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61d07b6-e95e-4e6f-871d-e074c1510a88 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:12.096825 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1055/2141] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:12 2022] PUT /identity/v3/projects/19733ce9e7f14e8da367fa7a04dfdc74/users/7fd0bf40927d4d86866e0763842a23b3/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:42:12.098671 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a11a2-1ab3-4a7c-b0b3-b8840ca317bc None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:12.099133 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a11a2-1ab3-4a7c-b0b3-b8840ca317bc None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:12.099248 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a11a2-1ab3-4a7c-b0b3-b8840ca317bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:12.129806 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-175a11a2-1ab3-4a7c-b0b3-b8840ca317bc None None] MFA Rules not processed for user `7fd0bf40927d4d86866e0763842a23b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:12.131306 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-175a11a2-1ab3-4a7c-b0b3-b8840ca317bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:12.219335 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1087/2142] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:42:12 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:12.231708 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61d07b6-e95e-4e6f-871d-e074c1510a88 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:12.233506 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-a61d07b6-e95e-4e6f-871d-e074c1510a88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:12.287304 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:12.310161 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61d07b6-e95e-4e6f-871d-e074c1510a88 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:12.318620 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ef368c-1f0b-4670-acd2-55e5d731b7a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:12.319310 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef368c-1f0b-4670-acd2-55e5d731b7a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:12.319600 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef368c-1f0b-4670-acd2-55e5d731b7a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:12.319884 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef368c-1f0b-4670-acd2-55e5d731b7a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:12.329968 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ef368c-1f0b-4670-acd2-55e5d731b7a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:12.331566 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ef368c-1f0b-4670-acd2-55e5d731b7a5 service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:12.349700 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1056/2143] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:12 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:42:13.198162 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175a11a2-1ab3-4a7c-b0b3-b8840ca317bc None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:13.201066 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-175a11a2-1ab3-4a7c-b0b3-b8840ca317bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:13.224146 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175a11a2-1ab3-4a7c-b0b3-b8840ca317bc None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:13.228339 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48afda0-c6d8-42ae-bc23-7fa5a9558afa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:13.229473 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48afda0-c6d8-42ae-bc23-7fa5a9558afa service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:13.230031 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48afda0-c6d8-42ae-bc23-7fa5a9558afa service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:13.230499 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48afda0-c6d8-42ae-bc23-7fa5a9558afa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:13.232619 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-f48afda0-c6d8-42ae-bc23-7fa5a9558afa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:13.319311 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48afda0-c6d8-42ae-bc23-7fa5a9558afa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:13.321097 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48afda0-c6d8-42ae-bc23-7fa5a9558afa service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:13.332845 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1088/2144] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:42:14.167720 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ef368c-1f0b-4670-acd2-55e5d731b7a5 service cinder] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:14.177762 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ef368c-1f0b-4670-acd2-55e5d731b7a5 service cinder] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:14.184515 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb59c4-693f-400c-8abc-09aa5b70a325 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:14.186038 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb59c4-693f-400c-8abc-09aa5b70a325 service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:14.186385 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb59c4-693f-400c-8abc-09aa5b70a325 service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:14.186669 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb59c4-693f-400c-8abc-09aa5b70a325 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:14.191870 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-f1fb59c4-693f-400c-8abc-09aa5b70a325 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:14.361984 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fb59c4-693f-400c-8abc-09aa5b70a325 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:14.363523 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fb59c4-693f-400c-8abc-09aa5b70a325 service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:14.387943 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1057/2145] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:14 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:42:15.122893 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48afda0-c6d8-42ae-bc23-7fa5a9558afa service cinder] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:15.125172 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-f48afda0-c6d8-42ae-bc23-7fa5a9558afa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:15.170564 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:15.211422 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48afda0-c6d8-42ae-bc23-7fa5a9558afa service cinder] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:15.217413 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426fe07b-2291-4f43-be33-6edf737c1387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:15.218345 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426fe07b-2291-4f43-be33-6edf737c1387 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:15.218633 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426fe07b-2291-4f43-be33-6edf737c1387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:15.218913 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426fe07b-2291-4f43-be33-6edf737c1387 admin admin] PATH_INFO: `/v3/users/c8215c41d0c54c62b4c9fea630481000` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:15.220152 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426fe07b-2291-4f43-be33-6edf737c1387 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8215c41d0c54c62b4c9fea630481000)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:15.222578 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426fe07b-2291-4f43-be33-6edf737c1387 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:15.278231 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-426fe07b-2291-4f43-be33-6edf737c1387 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8215c41d0c54c62b4c9fea630481000'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:15.290871 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-426fe07b-2291-4f43-be33-6edf737c1387 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8215c41d0c54c62b4c9fea630481000'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:15.311428 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-426fe07b-2291-4f43-be33-6edf737c1387 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8215c41d0c54c62b4c9fea630481000'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:15.312068 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-426fe07b-2291-4f43-be33-6edf737c1387 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8215c41d0c54c62b4c9fea630481000'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:15.312688 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-426fe07b-2291-4f43-be33-6edf737c1387 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8215c41d0c54c62b4c9fea630481000'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:15.326692 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1089/2146] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:42:15 2022] DELETE /identity/v3/users/c8215c41d0c54c62b4c9fea630481000 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:15.510879 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb59c4-693f-400c-8abc-09aa5b70a325 service cinder] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:15.523079 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb59c4-693f-400c-8abc-09aa5b70a325 service cinder] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:15.582417 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba19827d-600c-4f97-9873-5802db314dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:15.583271 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba19827d-600c-4f97-9873-5802db314dfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:15.583552 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba19827d-600c-4f97-9873-5802db314dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:15.583831 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba19827d-600c-4f97-9873-5802db314dfb admin admin] PATH_INFO: `/v3/projects/147d9b38796148cb996e8c61d1586b61` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:15.584997 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba19827d-600c-4f97-9873-5802db314dfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=147d9b38796148cb996e8c61d1586b61)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:15.586941 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba19827d-600c-4f97-9873-5802db314dfb admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:15.635343 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-ba19827d-600c-4f97-9873-5802db314dfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '147d9b38796148cb996e8c61d1586b61', 'request_id': 'req-ba19827d-600c-4f97-9873-5802db314dfb'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:15.641467 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-ba19827d-600c-4f97-9873-5802db314dfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '147d9b38796148cb996e8c61d1586b61', 'request_id': 'req-ba19827d-600c-4f97-9873-5802db314dfb'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:15.647554 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-ba19827d-600c-4f97-9873-5802db314dfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '147d9b38796148cb996e8c61d1586b61', 'request_id': 'req-ba19827d-600c-4f97-9873-5802db314dfb'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:15.660705 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-ba19827d-600c-4f97-9873-5802db314dfb admin admin] The token cache is being invalidate because project 147d9b38796148cb996e8c61d1586b61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:42:15.661000 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-ba19827d-600c-4f97-9873-5802db314dfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:15.661652 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-ba19827d-600c-4f97-9873-5802db314dfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:15.663812 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1058/2147] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:42:15 2022] DELETE /identity/v3/projects/147d9b38796148cb996e8c61d1586b61 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:42:15.668824 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cacd6c-6faa-46ec-8fa7-5d869368b008 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:15.669203 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cacd6c-6faa-46ec-8fa7-5d869368b008 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:15.669480 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cacd6c-6faa-46ec-8fa7-5d869368b008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:15.693409 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-b4cacd6c-6faa-46ec-8fa7-5d869368b008 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:15.694666 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b4cacd6c-6faa-46ec-8fa7-5d869368b008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:15.799719 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1090/2148] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:42:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:42:15.800610 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af996e8-5f1e-4c28-a3fc-1ee4dbff699e None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:15.801073 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af996e8-5f1e-4c28-a3fc-1ee4dbff699e None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:15.802163 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af996e8-5f1e-4c28-a3fc-1ee4dbff699e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:15.812730 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:15.826428 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-8af996e8-5f1e-4c28-a3fc-1ee4dbff699e None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:15.829058 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-8af996e8-5f1e-4c28-a3fc-1ee4dbff699e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:15.841068 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1059/2149] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:42:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:42:15.845331 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cacd6c-6faa-46ec-8fa7-5d869368b008 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:15.858017 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cacd6c-6faa-46ec-8fa7-5d869368b008 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:15.864154 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d71805-d393-4835-a475-b29205a2b2ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:15.864923 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d71805-d393-4835-a475-b29205a2b2ab admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:15.865343 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d71805-d393-4835-a475-b29205a2b2ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:15.866330 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d71805-d393-4835-a475-b29205a2b2ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:15.867172 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d71805-d393-4835-a475-b29205a2b2ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:42:15.867600 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d71805-d393-4835-a475-b29205a2b2ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:15.869340 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d71805-d393-4835-a475-b29205a2b2ab admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:15.888534 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1091/2150] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:42:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:42:15.889958 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af996e8-5f1e-4c28-a3fc-1ee4dbff699e None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:15.903658 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af996e8-5f1e-4c28-a3fc-1ee4dbff699e None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:15.909734 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5977d11-ec73-4a58-b910-69e96fb5e3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:15.910384 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5977d11-ec73-4a58-b910-69e96fb5e3dc admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:15.910952 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5977d11-ec73-4a58-b910-69e96fb5e3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:15.910952 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5977d11-ec73-4a58-b910-69e96fb5e3dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:15.911721 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5977d11-ec73-4a58-b910-69e96fb5e3dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:15.913590 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5977d11-ec73-4a58-b910-69e96fb5e3dc admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:15.953737 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1060/2151] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:42:15 2022] POST /identity/v3/projects => generated 383 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:15.954025 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d71805-d393-4835-a475-b29205a2b2ab admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:15.969787 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d71805-d393-4835-a475-b29205a2b2ab admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:16.028335 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:16.056433 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a32a959-0d87-4772-ac76-e26db10933e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:16.057237 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32a959-0d87-4772-ac76-e26db10933e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:16.057599 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32a959-0d87-4772-ac76-e26db10933e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:16.058056 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32a959-0d87-4772-ac76-e26db10933e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:16.058816 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a32a959-0d87-4772-ac76-e26db10933e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:16.060795 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a32a959-0d87-4772-ac76-e26db10933e8 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:16.081638 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1092/2152] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:42:15 2022] POST /identity/v3/users => generated 292 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:16.085765 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5977d11-ec73-4a58-b910-69e96fb5e3dc admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:16.096858 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5977d11-ec73-4a58-b910-69e96fb5e3dc admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:16.102493 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa98bea-1200-447e-8323-bd5b0983d7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:16.103340 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa98bea-1200-447e-8323-bd5b0983d7eb admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:16.103549 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa98bea-1200-447e-8323-bd5b0983d7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:16.103807 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa98bea-1200-447e-8323-bd5b0983d7eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:16.105158 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa98bea-1200-447e-8323-bd5b0983d7eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:16.106865 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa98bea-1200-447e-8323-bd5b0983d7eb admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:16.121745 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1061/2153] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:16.126126 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a32a959-0d87-4772-ac76-e26db10933e8 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:16.138432 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a32a959-0d87-4772-ac76-e26db10933e8 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:16.143648 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c3c4e9-789c-47bb-9d86-9dc300f8c002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:16.144299 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c3c4e9-789c-47bb-9d86-9dc300f8c002 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:16.144585 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c3c4e9-789c-47bb-9d86-9dc300f8c002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:16.144863 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c3c4e9-789c-47bb-9d86-9dc300f8c002 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:16.145595 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c3c4e9-789c-47bb-9d86-9dc300f8c002 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:16.147227 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c3c4e9-789c-47bb-9d86-9dc300f8c002 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:16.158095 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1093/2154] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:16.162486 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa98bea-1200-447e-8323-bd5b0983d7eb admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:16.173821 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa98bea-1200-447e-8323-bd5b0983d7eb admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:16.180395 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78090f84-2509-4796-a6f6-b114c5ac8903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:16.181975 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78090f84-2509-4796-a6f6-b114c5ac8903 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:16.182430 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78090f84-2509-4796-a6f6-b114c5ac8903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:16.182853 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78090f84-2509-4796-a6f6-b114c5ac8903 admin admin] PATH_INFO: `/v3/projects/68f7c00901424227acc04db187c3ca4d/users/fa7bf9a7cf0e49acb6ea3f90a4d29df2/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:16.201160 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78090f84-2509-4796-a6f6-b114c5ac8903 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68f7c00901424227acc04db187c3ca4d, user_id=fa7bf9a7cf0e49acb6ea3f90a4d29df2, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:16.203869 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78090f84-2509-4796-a6f6-b114c5ac8903 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:16.217636 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1062/2155] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:16 2022] PUT /identity/v3/projects/68f7c00901424227acc04db187c3ca4d/users/fa7bf9a7cf0e49acb6ea3f90a4d29df2/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:16.228499 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa033efc-ce9d-4aeb-a750-747480c36aa7 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:16.228798 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa033efc-ce9d-4aeb-a750-747480c36aa7 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:16.229115 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa033efc-ce9d-4aeb-a750-747480c36aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:16.257227 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-aa033efc-ce9d-4aeb-a750-747480c36aa7 None None] MFA Rules not processed for user `fa7bf9a7cf0e49acb6ea3f90a4d29df2`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:16.258490 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-aa033efc-ce9d-4aeb-a750-747480c36aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:16.327320 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1094/2156] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:42:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:16.557216 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78090f84-2509-4796-a6f6-b114c5ac8903 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:16.561656 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-78090f84-2509-4796-a6f6-b114c5ac8903 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:16.645135 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78090f84-2509-4796-a6f6-b114c5ac8903 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:16.650749 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797662f4-fcb1-4a60-9211-2101f3512b70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74bb1bc6821b4500b26f160ccae5b7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:16.651397 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797662f4-fcb1-4a60-9211-2101f3512b70 service nova] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:16.651683 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797662f4-fcb1-4a60-9211-2101f3512b70 service nova] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:16.651961 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797662f4-fcb1-4a60-9211-2101f3512b70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:16.660966 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797662f4-fcb1-4a60-9211-2101f3512b70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:16.662600 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797662f4-fcb1-4a60-9211-2101f3512b70 service nova] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:16.672110 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1063/2157] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:42:18.544336 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa033efc-ce9d-4aeb-a750-747480c36aa7 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:18.546038 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-aa033efc-ce9d-4aeb-a750-747480c36aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:18.628533 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa033efc-ce9d-4aeb-a750-747480c36aa7 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:18.632968 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa354f1b-e148-45ce-84f4-57673cff76a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:18.633872 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa354f1b-e148-45ce-84f4-57673cff76a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:18.634155 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa354f1b-e148-45ce-84f4-57673cff76a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:18.634425 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa354f1b-e148-45ce-84f4-57673cff76a4 admin admin] PATH_INFO: `/v3/users/c27f59913abc4b8fa7b9dd9a50f6f4a4` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:18.635610 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa354f1b-e148-45ce-84f4-57673cff76a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c27f59913abc4b8fa7b9dd9a50f6f4a4)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:18.637528 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa354f1b-e148-45ce-84f4-57673cff76a4 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:18.685048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-aa354f1b-e148-45ce-84f4-57673cff76a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c27f59913abc4b8fa7b9dd9a50f6f4a4'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:18.686182 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:18.699301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-aa354f1b-e148-45ce-84f4-57673cff76a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c27f59913abc4b8fa7b9dd9a50f6f4a4'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:18.706320 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-aa354f1b-e148-45ce-84f4-57673cff76a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c27f59913abc4b8fa7b9dd9a50f6f4a4'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:18.706957 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-aa354f1b-e148-45ce-84f4-57673cff76a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c27f59913abc4b8fa7b9dd9a50f6f4a4'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:18.707566 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-aa354f1b-e148-45ce-84f4-57673cff76a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c27f59913abc4b8fa7b9dd9a50f6f4a4'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:18.722672 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1095/2158] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:42:18 2022] DELETE /identity/v3/users/c27f59913abc4b8fa7b9dd9a50f6f4a4 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:18.728266 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9bebcf-7540-4f89-9837-c9cde8d1df70 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:18.728851 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9bebcf-7540-4f89-9837-c9cde8d1df70 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:18.728851 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9bebcf-7540-4f89-9837-c9cde8d1df70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:18.747400 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-cf9bebcf-7540-4f89-9837-c9cde8d1df70 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:18.748907 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-cf9bebcf-7540-4f89-9837-c9cde8d1df70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:18.808747 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1064/2159] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:42:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:18.818529 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa354f1b-e148-45ce-84f4-57673cff76a4 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:18.820017 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-aa354f1b-e148-45ce-84f4-57673cff76a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:18.889545 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa354f1b-e148-45ce-84f4-57673cff76a4 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:18.893777 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110de4fc-e7bc-4f7d-a9d1-f4bd7021d35d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:18.894435 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110de4fc-e7bc-4f7d-a9d1-f4bd7021d35d service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:18.894721 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110de4fc-e7bc-4f7d-a9d1-f4bd7021d35d service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:18.895000 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110de4fc-e7bc-4f7d-a9d1-f4bd7021d35d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:18.904908 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110de4fc-e7bc-4f7d-a9d1-f4bd7021d35d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:18.906394 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110de4fc-e7bc-4f7d-a9d1-f4bd7021d35d service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:18.917842 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1096/2160] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:42:19.225536 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9bebcf-7540-4f89-9837-c9cde8d1df70 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:19.227000 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-cf9bebcf-7540-4f89-9837-c9cde8d1df70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:19.240898 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9bebcf-7540-4f89-9837-c9cde8d1df70 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:19.245705 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5968cc56-f95f-48bb-aaf0-063bfa3c3fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:19.246575 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968cc56-f95f-48bb-aaf0-063bfa3c3fc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:19.246863 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968cc56-f95f-48bb-aaf0-063bfa3c3fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:19.247140 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968cc56-f95f-48bb-aaf0-063bfa3c3fc3 admin admin] PATH_INFO: `/v3/projects/001f8e102921445ba9cb4bbd17777a06` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:19.248602 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5968cc56-f95f-48bb-aaf0-063bfa3c3fc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=001f8e102921445ba9cb4bbd17777a06)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:19.250569 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5968cc56-f95f-48bb-aaf0-063bfa3c3fc3 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:19.280886 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:19.312493 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-5968cc56-f95f-48bb-aaf0-063bfa3c3fc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '001f8e102921445ba9cb4bbd17777a06', 'request_id': 'req-5968cc56-f95f-48bb-aaf0-063bfa3c3fc3'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:19.319594 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-5968cc56-f95f-48bb-aaf0-063bfa3c3fc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '001f8e102921445ba9cb4bbd17777a06', 'request_id': 'req-5968cc56-f95f-48bb-aaf0-063bfa3c3fc3'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:19.325241 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-5968cc56-f95f-48bb-aaf0-063bfa3c3fc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '001f8e102921445ba9cb4bbd17777a06', 'request_id': 'req-5968cc56-f95f-48bb-aaf0-063bfa3c3fc3'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:19.338550 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-5968cc56-f95f-48bb-aaf0-063bfa3c3fc3 admin admin] The token cache is being invalidate because project 001f8e102921445ba9cb4bbd17777a06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:42:19.338921 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-5968cc56-f95f-48bb-aaf0-063bfa3c3fc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:19.340003 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-5968cc56-f95f-48bb-aaf0-063bfa3c3fc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:19.342935 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1065/2161] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:42:19 2022] DELETE /identity/v3/projects/001f8e102921445ba9cb4bbd17777a06 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:19.348175 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110de4fc-e7bc-4f7d-a9d1-f4bd7021d35d service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:19.358226 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110de4fc-e7bc-4f7d-a9d1-f4bd7021d35d service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:19.416147 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc137b11-52d9-4643-a2ea-48f903eafc20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:19.417319 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc137b11-52d9-4643-a2ea-48f903eafc20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:19.417784 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc137b11-52d9-4643-a2ea-48f903eafc20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:19.418213 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc137b11-52d9-4643-a2ea-48f903eafc20 admin admin] PATH_INFO: `/v3/users/d2219a6b9ad44796912db59b0bffad9e` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:19.419687 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc137b11-52d9-4643-a2ea-48f903eafc20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2219a6b9ad44796912db59b0bffad9e)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:19.421828 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc137b11-52d9-4643-a2ea-48f903eafc20 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:19.475766 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-bc137b11-52d9-4643-a2ea-48f903eafc20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2219a6b9ad44796912db59b0bffad9e'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:19.487841 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-bc137b11-52d9-4643-a2ea-48f903eafc20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2219a6b9ad44796912db59b0bffad9e'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:19.491485 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:19.495789 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-bc137b11-52d9-4643-a2ea-48f903eafc20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2219a6b9ad44796912db59b0bffad9e'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:19.496886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-bc137b11-52d9-4643-a2ea-48f903eafc20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2219a6b9ad44796912db59b0bffad9e'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:19.497624 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-bc137b11-52d9-4643-a2ea-48f903eafc20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2219a6b9ad44796912db59b0bffad9e'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:19.511552 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1097/2162] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:42:19 2022] DELETE /identity/v3/users/d2219a6b9ad44796912db59b0bffad9e => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:19.798097 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5968cc56-f95f-48bb-aaf0-063bfa3c3fc3 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:19.799902 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-5968cc56-f95f-48bb-aaf0-063bfa3c3fc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:19.812677 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5968cc56-f95f-48bb-aaf0-063bfa3c3fc3 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:19.871483 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc912dc-2c03-41c7-9b50-cfa7ff8cb5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:19.872386 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc912dc-2c03-41c7-9b50-cfa7ff8cb5e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:19.872929 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc912dc-2c03-41c7-9b50-cfa7ff8cb5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:19.873423 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc912dc-2c03-41c7-9b50-cfa7ff8cb5e0 admin admin] PATH_INFO: `/v3/projects/c30869954ce048cd86f483c8a1735750` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:19.874937 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc912dc-2c03-41c7-9b50-cfa7ff8cb5e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c30869954ce048cd86f483c8a1735750)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:19.876925 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc912dc-2c03-41c7-9b50-cfa7ff8cb5e0 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:19.924112 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-3bc912dc-2c03-41c7-9b50-cfa7ff8cb5e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c30869954ce048cd86f483c8a1735750', 'request_id': 'req-3bc912dc-2c03-41c7-9b50-cfa7ff8cb5e0'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:19.930623 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-3bc912dc-2c03-41c7-9b50-cfa7ff8cb5e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c30869954ce048cd86f483c8a1735750', 'request_id': 'req-3bc912dc-2c03-41c7-9b50-cfa7ff8cb5e0'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:19.936162 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-3bc912dc-2c03-41c7-9b50-cfa7ff8cb5e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c30869954ce048cd86f483c8a1735750', 'request_id': 'req-3bc912dc-2c03-41c7-9b50-cfa7ff8cb5e0'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:19.949172 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-3bc912dc-2c03-41c7-9b50-cfa7ff8cb5e0 admin admin] The token cache is being invalidate because project c30869954ce048cd86f483c8a1735750 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:42:19.949458 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-3bc912dc-2c03-41c7-9b50-cfa7ff8cb5e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:19.950084 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-3bc912dc-2c03-41c7-9b50-cfa7ff8cb5e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:19.951937 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1066/2163] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:42:19 2022] DELETE /identity/v3/projects/c30869954ce048cd86f483c8a1735750 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:19.956815 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc137b11-52d9-4643-a2ea-48f903eafc20 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:19.958369 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-bc137b11-52d9-4643-a2ea-48f903eafc20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:19.971522 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc137b11-52d9-4643-a2ea-48f903eafc20 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:20.030029 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfa4cf5-515d-43f4-8141-762231c76334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:20.030906 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa4cf5-515d-43f4-8141-762231c76334 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:20.031192 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa4cf5-515d-43f4-8141-762231c76334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:20.031470 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa4cf5-515d-43f4-8141-762231c76334 admin admin] PATH_INFO: `/v3/users/5fca590160504ca88d67ee507616f126` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:20.032691 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfa4cf5-515d-43f4-8141-762231c76334 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fca590160504ca88d67ee507616f126)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:20.034815 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfa4cf5-515d-43f4-8141-762231c76334 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:20.094545 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-abfa4cf5-515d-43f4-8141-762231c76334 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fca590160504ca88d67ee507616f126'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:20.114895 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-abfa4cf5-515d-43f4-8141-762231c76334 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fca590160504ca88d67ee507616f126'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:20.127861 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-abfa4cf5-515d-43f4-8141-762231c76334 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fca590160504ca88d67ee507616f126'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:20.129536 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-abfa4cf5-515d-43f4-8141-762231c76334 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fca590160504ca88d67ee507616f126'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:20.130462 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-abfa4cf5-515d-43f4-8141-762231c76334 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fca590160504ca88d67ee507616f126'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:20.148167 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1098/2164] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:42:19 2022] DELETE /identity/v3/users/5fca590160504ca88d67ee507616f126 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:20.447415 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc912dc-2c03-41c7-9b50-cfa7ff8cb5e0 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:20.448848 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-3bc912dc-2c03-41c7-9b50-cfa7ff8cb5e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:20.462786 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc912dc-2c03-41c7-9b50-cfa7ff8cb5e0 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:20.506619 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:20.531055 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be65c8f-ad8f-4e43-a928-abd2fb7d4530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:20.531918 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be65c8f-ad8f-4e43-a928-abd2fb7d4530 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:20.532205 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be65c8f-ad8f-4e43-a928-abd2fb7d4530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:20.532478 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be65c8f-ad8f-4e43-a928-abd2fb7d4530 admin admin] PATH_INFO: `/v3/projects/a2ce5f44201e484ebb88033bf3f8c65f` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:20.534419 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be65c8f-ad8f-4e43-a928-abd2fb7d4530 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2ce5f44201e484ebb88033bf3f8c65f)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:20.536331 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be65c8f-ad8f-4e43-a928-abd2fb7d4530 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:20.584146 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-0be65c8f-ad8f-4e43-a928-abd2fb7d4530 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2ce5f44201e484ebb88033bf3f8c65f', 'request_id': 'req-0be65c8f-ad8f-4e43-a928-abd2fb7d4530'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:20.590523 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-0be65c8f-ad8f-4e43-a928-abd2fb7d4530 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2ce5f44201e484ebb88033bf3f8c65f', 'request_id': 'req-0be65c8f-ad8f-4e43-a928-abd2fb7d4530'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:20.597018 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-0be65c8f-ad8f-4e43-a928-abd2fb7d4530 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2ce5f44201e484ebb88033bf3f8c65f', 'request_id': 'req-0be65c8f-ad8f-4e43-a928-abd2fb7d4530'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:20.610596 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-0be65c8f-ad8f-4e43-a928-abd2fb7d4530 admin admin] The token cache is being invalidate because project a2ce5f44201e484ebb88033bf3f8c65f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:42:20.610889 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-0be65c8f-ad8f-4e43-a928-abd2fb7d4530 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:20.612115 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-0be65c8f-ad8f-4e43-a928-abd2fb7d4530 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:20.613994 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1067/2165] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:42:20 2022] DELETE /identity/v3/projects/a2ce5f44201e484ebb88033bf3f8c65f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:20.620584 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da897415-5c43-4f61-a924-01158fe0cefd None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:20.620584 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da897415-5c43-4f61-a924-01158fe0cefd None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:20.620584 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da897415-5c43-4f61-a924-01158fe0cefd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:20.637167 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-da897415-5c43-4f61-a924-01158fe0cefd None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:20.638421 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-da897415-5c43-4f61-a924-01158fe0cefd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:20.704535 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1099/2166] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:42:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:20.713497 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c3eafa-f860-4d12-87cf-4109f9591629 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:20.713819 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c3eafa-f860-4d12-87cf-4109f9591629 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:20.714101 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c3eafa-f860-4d12-87cf-4109f9591629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:20.734308 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-a0c3eafa-f860-4d12-87cf-4109f9591629 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:20.735533 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-a0c3eafa-f860-4d12-87cf-4109f9591629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:20.744307 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1068/2167] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:42:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:20.748767 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da897415-5c43-4f61-a924-01158fe0cefd None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:20.758881 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da897415-5c43-4f61-a924-01158fe0cefd None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:20.763398 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf159f-5e80-4fe8-97df-b6eecadddf92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:20.764047 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf159f-5e80-4fe8-97df-b6eecadddf92 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:20.764334 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf159f-5e80-4fe8-97df-b6eecadddf92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:20.764610 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf159f-5e80-4fe8-97df-b6eecadddf92 admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:20.765334 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cf159f-5e80-4fe8-97df-b6eecadddf92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:42:20.765642 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cf159f-5e80-4fe8-97df-b6eecadddf92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:20.767273 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cf159f-5e80-4fe8-97df-b6eecadddf92 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:20.769791 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:20.781635 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1100/2168] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:42:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:42:20.786389 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c3eafa-f860-4d12-87cf-4109f9591629 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:20.795783 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c3eafa-f860-4d12-87cf-4109f9591629 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:20.800639 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd20baf4-7b38-4bc2-991b-ba14317161fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:20.801321 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd20baf4-7b38-4bc2-991b-ba14317161fe admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:20.801608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd20baf4-7b38-4bc2-991b-ba14317161fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:20.801920 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd20baf4-7b38-4bc2-991b-ba14317161fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:20.802620 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd20baf4-7b38-4bc2-991b-ba14317161fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:20.804525 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd20baf4-7b38-4bc2-991b-ba14317161fe admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:20.831135 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1069/2169] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:42:20 2022] POST /identity/v3/projects => generated 389 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:20.834884 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf159f-5e80-4fe8-97df-b6eecadddf92 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:20.845222 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf159f-5e80-4fe8-97df-b6eecadddf92 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:20.902523 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d805af92-c8d6-4851-9bd8-12cf5be1d637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:20.903209 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805af92-c8d6-4851-9bd8-12cf5be1d637 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:20.903493 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805af92-c8d6-4851-9bd8-12cf5be1d637 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:20.903771 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805af92-c8d6-4851-9bd8-12cf5be1d637 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:20.904460 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d805af92-c8d6-4851-9bd8-12cf5be1d637 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:20.906411 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d805af92-c8d6-4851-9bd8-12cf5be1d637 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:20.926559 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1101/2170] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:42:20 2022] POST /identity/v3/users => generated 295 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:20.930676 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd20baf4-7b38-4bc2-991b-ba14317161fe admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:20.941134 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd20baf4-7b38-4bc2-991b-ba14317161fe admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:20.945876 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d687c110-37c2-479b-8755-e0ffd92ebd80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:20.946542 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687c110-37c2-479b-8755-e0ffd92ebd80 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:20.946833 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687c110-37c2-479b-8755-e0ffd92ebd80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:20.947115 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687c110-37c2-479b-8755-e0ffd92ebd80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:20.947847 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d687c110-37c2-479b-8755-e0ffd92ebd80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:20.949498 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d687c110-37c2-479b-8755-e0ffd92ebd80 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:20.961826 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1070/2171] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:20.965930 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d805af92-c8d6-4851-9bd8-12cf5be1d637 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:20.981046 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d805af92-c8d6-4851-9bd8-12cf5be1d637 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:20.986613 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6dd384-b1f4-428a-a03b-9197b0e4b8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:20.987806 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6dd384-b1f4-428a-a03b-9197b0e4b8dc admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:20.988251 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6dd384-b1f4-428a-a03b-9197b0e4b8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:20.988539 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6dd384-b1f4-428a-a03b-9197b0e4b8dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:20.989287 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6dd384-b1f4-428a-a03b-9197b0e4b8dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:20.990935 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6dd384-b1f4-428a-a03b-9197b0e4b8dc admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:21.004357 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1102/2172] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:20 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:21.008600 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d687c110-37c2-479b-8755-e0ffd92ebd80 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:21.019359 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d687c110-37c2-479b-8755-e0ffd92ebd80 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:21.023937 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa84dd3-202e-4a23-ba24-f50aaa28cf82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:21.024802 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa84dd3-202e-4a23-ba24-f50aaa28cf82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:21.025086 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa84dd3-202e-4a23-ba24-f50aaa28cf82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:21.025361 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa84dd3-202e-4a23-ba24-f50aaa28cf82 admin admin] PATH_INFO: `/v3/projects/1431292bdb224b098c38d8dbf641d9a3/users/3630aa55e83f4c70a6780d5f7414374a/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:21.040476 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa84dd3-202e-4a23-ba24-f50aaa28cf82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1431292bdb224b098c38d8dbf641d9a3, user_id=3630aa55e83f4c70a6780d5f7414374a, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:21.043193 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa84dd3-202e-4a23-ba24-f50aaa28cf82 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:21.057524 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1071/2173] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:21 2022] PUT /identity/v3/projects/1431292bdb224b098c38d8dbf641d9a3/users/3630aa55e83f4c70a6780d5f7414374a/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:21.068882 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f7fc6-4872-4172-a7b1-cb87ebb46c2d None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:21.069199 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f7fc6-4872-4172-a7b1-cb87ebb46c2d None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:21.069478 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f7fc6-4872-4172-a7b1-cb87ebb46c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:21.100313 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-615f7fc6-4872-4172-a7b1-cb87ebb46c2d None None] MFA Rules not processed for user `3630aa55e83f4c70a6780d5f7414374a`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:21.101545 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-615f7fc6-4872-4172-a7b1-cb87ebb46c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:21.166140 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1103/2174] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:42:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:21.176863 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa84dd3-202e-4a23-ba24-f50aaa28cf82 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:21.178570 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-2fa84dd3-202e-4a23-ba24-f50aaa28cf82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:21.239795 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa84dd3-202e-4a23-ba24-f50aaa28cf82 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:21.243782 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1f53cb-2d14-4264-bcb6-ea456d1e38cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:21.244451 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f53cb-2d14-4264-bcb6-ea456d1e38cd service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:21.244735 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f53cb-2d14-4264-bcb6-ea456d1e38cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:21.245009 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f53cb-2d14-4264-bcb6-ea456d1e38cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:21.256241 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1f53cb-2d14-4264-bcb6-ea456d1e38cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:21.257981 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1f53cb-2d14-4264-bcb6-ea456d1e38cd service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:21.268705 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1072/2175] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:21 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:42:29.409899 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615f7fc6-4872-4172-a7b1-cb87ebb46c2d None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:29.409899 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-615f7fc6-4872-4172-a7b1-cb87ebb46c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:29.477664 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615f7fc6-4872-4172-a7b1-cb87ebb46c2d None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:29.482286 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdae5a5b-a735-41a7-a837-2da966dd4ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:29.483311 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae5a5b-a735-41a7-a837-2da966dd4ab8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:29.483755 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae5a5b-a735-41a7-a837-2da966dd4ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:29.484119 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae5a5b-a735-41a7-a837-2da966dd4ab8 admin admin] PATH_INFO: `/v3/users/fa7bf9a7cf0e49acb6ea3f90a4d29df2` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:29.485349 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdae5a5b-a735-41a7-a837-2da966dd4ab8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa7bf9a7cf0e49acb6ea3f90a4d29df2)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:29.487950 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdae5a5b-a735-41a7-a837-2da966dd4ab8 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:29.490799 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:29.586858 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-fdae5a5b-a735-41a7-a837-2da966dd4ab8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa7bf9a7cf0e49acb6ea3f90a4d29df2'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:29.609783 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-fdae5a5b-a735-41a7-a837-2da966dd4ab8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa7bf9a7cf0e49acb6ea3f90a4d29df2'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:29.621576 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-fdae5a5b-a735-41a7-a837-2da966dd4ab8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa7bf9a7cf0e49acb6ea3f90a4d29df2'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:29.622939 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-fdae5a5b-a735-41a7-a837-2da966dd4ab8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa7bf9a7cf0e49acb6ea3f90a4d29df2'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:29.624015 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-fdae5a5b-a735-41a7-a837-2da966dd4ab8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa7bf9a7cf0e49acb6ea3f90a4d29df2'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:29.649753 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1104/2176] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:42:29 2022] DELETE /identity/v3/users/fa7bf9a7cf0e49acb6ea3f90a4d29df2 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:42:29.655171 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c3b0d-a8ff-41ca-8c4d-2a67f02296a2 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:29.655690 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c3b0d-a8ff-41ca-8c4d-2a67f02296a2 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:29.655821 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c3b0d-a8ff-41ca-8c4d-2a67f02296a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:29.688993 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-3e8c3b0d-a8ff-41ca-8c4d-2a67f02296a2 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:29.691030 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-3e8c3b0d-a8ff-41ca-8c4d-2a67f02296a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:29.697984 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:29.774610 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1073/2177] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:42:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:29.789592 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdae5a5b-a735-41a7-a837-2da966dd4ab8 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:29.791079 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-fdae5a5b-a735-41a7-a837-2da966dd4ab8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:29.882035 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdae5a5b-a735-41a7-a837-2da966dd4ab8 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:29.886344 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bacf80-b2d3-43c0-9334-9ad229f5d36f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:29.887188 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bacf80-b2d3-43c0-9334-9ad229f5d36f service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:29.887535 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bacf80-b2d3-43c0-9334-9ad229f5d36f service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:29.887838 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bacf80-b2d3-43c0-9334-9ad229f5d36f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:29.899948 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bacf80-b2d3-43c0-9334-9ad229f5d36f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:29.901426 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bacf80-b2d3-43c0-9334-9ad229f5d36f service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:29.912285 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1105/2178] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:42:30.215352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8c3b0d-a8ff-41ca-8c4d-2a67f02296a2 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:30.216886 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-3e8c3b0d-a8ff-41ca-8c4d-2a67f02296a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:30.235844 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8c3b0d-a8ff-41ca-8c4d-2a67f02296a2 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:30.249261 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3d450d-d859-4b35-b3c9-c3706b6e75b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:30.250176 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d450d-d859-4b35-b3c9-c3706b6e75b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:30.250461 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d450d-d859-4b35-b3c9-c3706b6e75b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:30.250756 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d450d-d859-4b35-b3c9-c3706b6e75b7 admin admin] PATH_INFO: `/v3/projects/68f7c00901424227acc04db187c3ca4d` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:30.252500 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3d450d-d859-4b35-b3c9-c3706b6e75b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68f7c00901424227acc04db187c3ca4d)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:30.256676 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3d450d-d859-4b35-b3c9-c3706b6e75b7 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:30.320364 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-2c3d450d-d859-4b35-b3c9-c3706b6e75b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68f7c00901424227acc04db187c3ca4d', 'request_id': 'req-2c3d450d-d859-4b35-b3c9-c3706b6e75b7'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:30.326627 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-2c3d450d-d859-4b35-b3c9-c3706b6e75b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68f7c00901424227acc04db187c3ca4d', 'request_id': 'req-2c3d450d-d859-4b35-b3c9-c3706b6e75b7'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:30.332594 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-2c3d450d-d859-4b35-b3c9-c3706b6e75b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68f7c00901424227acc04db187c3ca4d', 'request_id': 'req-2c3d450d-d859-4b35-b3c9-c3706b6e75b7'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:30.344899 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-2c3d450d-d859-4b35-b3c9-c3706b6e75b7 admin admin] The token cache is being invalidate because project 68f7c00901424227acc04db187c3ca4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:42:30.345184 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-2c3d450d-d859-4b35-b3c9-c3706b6e75b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:30.346930 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-2c3d450d-d859-4b35-b3c9-c3706b6e75b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:30.348491 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1074/2179] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:42:30 2022] DELETE /identity/v3/projects/68f7c00901424227acc04db187c3ca4d => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:30.353480 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ed691-e0ac-4967-aef7-2ed2de4c5ec9 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:30.353811 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ed691-e0ac-4967-aef7-2ed2de4c5ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:30.354088 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ed691-e0ac-4967-aef7-2ed2de4c5ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:30.372987 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-148ed691-e0ac-4967-aef7-2ed2de4c5ec9 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:30.374228 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-148ed691-e0ac-4967-aef7-2ed2de4c5ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:30.434344 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1106/2180] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:42:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:30.442315 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab88dc08-46ba-432f-89a0-a75d423c7118 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:30.442605 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab88dc08-46ba-432f-89a0-a75d423c7118 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:30.442885 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab88dc08-46ba-432f-89a0-a75d423c7118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:30.462576 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-ab88dc08-46ba-432f-89a0-a75d423c7118 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:30.463772 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-ab88dc08-46ba-432f-89a0-a75d423c7118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:30.472421 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1075/2181] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:42:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:30.476584 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ed691-e0ac-4967-aef7-2ed2de4c5ec9 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:30.487107 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ed691-e0ac-4967-aef7-2ed2de4c5ec9 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:30.491932 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e3701-78c4-4d17-83f3-92ab8af9a79f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:30.492600 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e3701-78c4-4d17-83f3-92ab8af9a79f admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:30.492885 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e3701-78c4-4d17-83f3-92ab8af9a79f admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:30.493164 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e3701-78c4-4d17-83f3-92ab8af9a79f admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:30.493938 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405e3701-78c4-4d17-83f3-92ab8af9a79f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:42:30.494248 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405e3701-78c4-4d17-83f3-92ab8af9a79f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:30.495866 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405e3701-78c4-4d17-83f3-92ab8af9a79f admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:30.508952 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1107/2182] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:42:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:42:30.513166 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab88dc08-46ba-432f-89a0-a75d423c7118 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:30.523309 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab88dc08-46ba-432f-89a0-a75d423c7118 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:30.527909 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a0ff28-1439-43be-8ff9-768a7ecd366e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:30.528562 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a0ff28-1439-43be-8ff9-768a7ecd366e admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:30.528844 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a0ff28-1439-43be-8ff9-768a7ecd366e admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:30.529134 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a0ff28-1439-43be-8ff9-768a7ecd366e admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:30.529844 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a0ff28-1439-43be-8ff9-768a7ecd366e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:30.531734 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a0ff28-1439-43be-8ff9-768a7ecd366e admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:30.557815 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1076/2183] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:42:30 2022] POST /identity/v3/projects => generated 363 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:30.561949 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e3701-78c4-4d17-83f3-92ab8af9a79f admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:30.572210 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e3701-78c4-4d17-83f3-92ab8af9a79f admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:30.581338 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:30.631077 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433ef058-762c-4c1f-8b76-a02ca207cfa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:30.631731 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ef058-762c-4c1f-8b76-a02ca207cfa3 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:30.632016 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ef058-762c-4c1f-8b76-a02ca207cfa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:30.632290 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ef058-762c-4c1f-8b76-a02ca207cfa3 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:30.632978 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433ef058-762c-4c1f-8b76-a02ca207cfa3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:30.634905 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433ef058-762c-4c1f-8b76-a02ca207cfa3 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:30.654909 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1108/2184] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:42:30 2022] POST /identity/v3/users => generated 282 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:30.659477 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a0ff28-1439-43be-8ff9-768a7ecd366e admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:30.669994 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a0ff28-1439-43be-8ff9-768a7ecd366e admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:30.674835 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4835d31f-7dc8-45af-88e5-63e105c102ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:30.675532 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835d31f-7dc8-45af-88e5-63e105c102ab admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:30.675818 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835d31f-7dc8-45af-88e5-63e105c102ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:30.676094 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835d31f-7dc8-45af-88e5-63e105c102ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:30.676816 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4835d31f-7dc8-45af-88e5-63e105c102ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:30.678491 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4835d31f-7dc8-45af-88e5-63e105c102ab admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:30.689349 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1077/2185] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:30.693427 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433ef058-762c-4c1f-8b76-a02ca207cfa3 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:30.703458 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433ef058-762c-4c1f-8b76-a02ca207cfa3 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:30.708800 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764fa640-fd7c-49ff-8061-1cf1f3d20116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:30.709453 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764fa640-fd7c-49ff-8061-1cf1f3d20116 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:30.709770 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764fa640-fd7c-49ff-8061-1cf1f3d20116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:30.710055 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764fa640-fd7c-49ff-8061-1cf1f3d20116 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:30.710774 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764fa640-fd7c-49ff-8061-1cf1f3d20116 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:30.712367 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764fa640-fd7c-49ff-8061-1cf1f3d20116 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:30.722970 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1109/2186] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:30.727119 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4835d31f-7dc8-45af-88e5-63e105c102ab admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:30.737755 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4835d31f-7dc8-45af-88e5-63e105c102ab admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:30.742803 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b3a994-2846-4782-87a3-01a138a1def5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:30.743673 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3a994-2846-4782-87a3-01a138a1def5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:30.743952 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3a994-2846-4782-87a3-01a138a1def5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:30.744228 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3a994-2846-4782-87a3-01a138a1def5 admin admin] PATH_INFO: `/v3/projects/8b9ff09d98a6448f97bacf0da45b4510/users/1672e4a297e943b6ad5752e9271a9fdf/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:30.759108 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b3a994-2846-4782-87a3-01a138a1def5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b9ff09d98a6448f97bacf0da45b4510, user_id=1672e4a297e943b6ad5752e9271a9fdf, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:30.761783 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b3a994-2846-4782-87a3-01a138a1def5 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:30.775951 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1078/2187] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:30 2022] PUT /identity/v3/projects/8b9ff09d98a6448f97bacf0da45b4510/users/1672e4a297e943b6ad5752e9271a9fdf/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:30.787132 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e5872-a0aa-4ed3-842c-504851e3ca5d None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:30.787423 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e5872-a0aa-4ed3-842c-504851e3ca5d None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:30.787702 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e5872-a0aa-4ed3-842c-504851e3ca5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:30.816288 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-397e5872-a0aa-4ed3-842c-504851e3ca5d None None] MFA Rules not processed for user `1672e4a297e943b6ad5752e9271a9fdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:30.817505 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-397e5872-a0aa-4ed3-842c-504851e3ca5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:30.888230 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1110/2188] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:42:30 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:30.891426 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b3a994-2846-4782-87a3-01a138a1def5 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:30.924639 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:30.965961 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b3a994-2846-4782-87a3-01a138a1def5 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:30.971213 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7886b7f-d129-48ec-98ba-844964bf7cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:30.971818 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7886b7f-d129-48ec-98ba-844964bf7cc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:30.972102 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7886b7f-d129-48ec-98ba-844964bf7cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:30.972379 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7886b7f-d129-48ec-98ba-844964bf7cc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:30.973627 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7886b7f-d129-48ec-98ba-844964bf7cc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:30.976374 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7886b7f-d129-48ec-98ba-844964bf7cc3 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:31.006493 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1079/2189] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:42:30 2022] POST /identity/v3/projects => generated 365 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:31.010777 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e5872-a0aa-4ed3-842c-504851e3ca5d None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:31.023340 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e5872-a0aa-4ed3-842c-504851e3ca5d None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:31.088935 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023851d4-0a8a-4c72-b316-b6c48f75aaa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:31.089616 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023851d4-0a8a-4c72-b316-b6c48f75aaa8 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:31.089939 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023851d4-0a8a-4c72-b316-b6c48f75aaa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:31.090283 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023851d4-0a8a-4c72-b316-b6c48f75aaa8 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:31.091047 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023851d4-0a8a-4c72-b316-b6c48f75aaa8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:31.093391 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023851d4-0a8a-4c72-b316-b6c48f75aaa8 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:31.114796 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1111/2190] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:42:31 2022] POST /identity/v3/users => generated 289 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:42:31.118698 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7886b7f-d129-48ec-98ba-844964bf7cc3 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:31.130012 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7886b7f-d129-48ec-98ba-844964bf7cc3 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:31.134743 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c33fb1-470b-4584-839d-7f15a511a16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:31.135406 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c33fb1-470b-4584-839d-7f15a511a16b admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:31.135694 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c33fb1-470b-4584-839d-7f15a511a16b admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:31.136025 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c33fb1-470b-4584-839d-7f15a511a16b admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:31.136688 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c33fb1-470b-4584-839d-7f15a511a16b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:31.138322 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c33fb1-470b-4584-839d-7f15a511a16b admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:31.149907 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1080/2191] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:31.154176 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023851d4-0a8a-4c72-b316-b6c48f75aaa8 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:31.164370 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023851d4-0a8a-4c72-b316-b6c48f75aaa8 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:31.169031 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f446bb78-95c4-49a7-b60c-d6bf19962707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:31.169956 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f446bb78-95c4-49a7-b60c-d6bf19962707 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:31.170238 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f446bb78-95c4-49a7-b60c-d6bf19962707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:31.170511 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f446bb78-95c4-49a7-b60c-d6bf19962707 admin admin] PATH_INFO: `/v3/projects/499728cb599a46e49b3966880476c243/users/59bbe0c75e6b4b969eecc1d50fc11965/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:31.173536 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:31.186692 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f446bb78-95c4-49a7-b60c-d6bf19962707 admin admin] RBAC: Authorizing `identity:create_grant(project_id=499728cb599a46e49b3966880476c243, user_id=59bbe0c75e6b4b969eecc1d50fc11965, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:31.189906 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f446bb78-95c4-49a7-b60c-d6bf19962707 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:31.202889 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1112/2192] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:31 2022] PUT /identity/v3/projects/499728cb599a46e49b3966880476c243/users/59bbe0c75e6b4b969eecc1d50fc11965/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:31.208016 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c33fb1-470b-4584-839d-7f15a511a16b admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:31.218352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c33fb1-470b-4584-839d-7f15a511a16b admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:31.276002 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a54b05-6009-4400-b063-c491e208ddb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:31.276664 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a54b05-6009-4400-b063-c491e208ddb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:31.276947 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a54b05-6009-4400-b063-c491e208ddb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:31.277218 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a54b05-6009-4400-b063-c491e208ddb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:31.277993 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a54b05-6009-4400-b063-c491e208ddb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:31.279683 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a54b05-6009-4400-b063-c491e208ddb4 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:31.289833 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1081/2193] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:31.294120 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f446bb78-95c4-49a7-b60c-d6bf19962707 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:31.305249 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f446bb78-95c4-49a7-b60c-d6bf19962707 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:31.310055 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fe70f2-da6e-47b8-ac22-f9555ebe6ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:31.311076 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe70f2-da6e-47b8-ac22-f9555ebe6ea7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:31.311223 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe70f2-da6e-47b8-ac22-f9555ebe6ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:31.311503 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe70f2-da6e-47b8-ac22-f9555ebe6ea7 admin admin] PATH_INFO: `/v3/projects/499728cb599a46e49b3966880476c243/users/59bbe0c75e6b4b969eecc1d50fc11965/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:31.313480 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fe70f2-da6e-47b8-ac22-f9555ebe6ea7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=499728cb599a46e49b3966880476c243, user_id=59bbe0c75e6b4b969eecc1d50fc11965, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:31.317122 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fe70f2-da6e-47b8-ac22-f9555ebe6ea7 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:31.327950 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1113/2194] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:31 2022] PUT /identity/v3/projects/499728cb599a46e49b3966880476c243/users/59bbe0c75e6b4b969eecc1d50fc11965/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:31.339361 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ffa737-3f75-4efe-9a9d-a70895469e38 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:31.339649 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ffa737-3f75-4efe-9a9d-a70895469e38 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:31.339950 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ffa737-3f75-4efe-9a9d-a70895469e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:31.368063 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-77ffa737-3f75-4efe-9a9d-a70895469e38 None None] MFA Rules not processed for user `59bbe0c75e6b4b969eecc1d50fc11965`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:31.369281 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-77ffa737-3f75-4efe-9a9d-a70895469e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:31.440407 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1082/2195] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:42:31 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:31.451501 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fe70f2-da6e-47b8-ac22-f9555ebe6ea7 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:31.453286 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-46fe70f2-da6e-47b8-ac22-f9555ebe6ea7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:31.510916 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fe70f2-da6e-47b8-ac22-f9555ebe6ea7 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:31.514561 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0cce95-7a08-4e4f-b724-4614c99668c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:31.515203 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0cce95-7a08-4e4f-b724-4614c99668c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:31.515489 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0cce95-7a08-4e4f-b724-4614c99668c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:31.515763 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0cce95-7a08-4e4f-b724-4614c99668c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:31.524793 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0cce95-7a08-4e4f-b724-4614c99668c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:31.526258 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0cce95-7a08-4e4f-b724-4614c99668c1 service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:31.535821 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1114/2196] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:31 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:42:31.619945 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ffa737-3f75-4efe-9a9d-a70895469e38 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:31.629993 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ffa737-3f75-4efe-9a9d-a70895469e38 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:31.633476 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f783e-27e6-466b-b1e4-be2ce33c4f86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:31.634186 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f783e-27e6-466b-b1e4-be2ce33c4f86 service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:31.634472 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f783e-27e6-466b-b1e4-be2ce33c4f86 service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:31.634852 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f783e-27e6-466b-b1e4-be2ce33c4f86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:31.655404 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:31.692181 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745f783e-27e6-466b-b1e4-be2ce33c4f86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:31.693651 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745f783e-27e6-466b-b1e4-be2ce33c4f86 service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:31.702706 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1083/2197] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:31 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:42:41.531682 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=37,keystone:INSERT=2 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:41.698766 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=17 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:45.292581 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0cce95-7a08-4e4f-b724-4614c99668c1 service cinder] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:45.294086 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-8b0cce95-7a08-4e4f-b724-4614c99668c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:45.387180 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0cce95-7a08-4e4f-b724-4614c99668c1 service cinder] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:45.392960 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe1751a-3c5a-48dc-86b7-93fbb6f7acd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:45.394083 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe1751a-3c5a-48dc-86b7-93fbb6f7acd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:45.394445 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe1751a-3c5a-48dc-86b7-93fbb6f7acd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:45.394785 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe1751a-3c5a-48dc-86b7-93fbb6f7acd5 admin admin] PATH_INFO: `/v3/users/3630aa55e83f4c70a6780d5f7414374a` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:45.396116 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe1751a-3c5a-48dc-86b7-93fbb6f7acd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3630aa55e83f4c70a6780d5f7414374a)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:45.398196 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe1751a-3c5a-48dc-86b7-93fbb6f7acd5 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:45.461919 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-dfe1751a-3c5a-48dc-86b7-93fbb6f7acd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3630aa55e83f4c70a6780d5f7414374a'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:45.479180 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-dfe1751a-3c5a-48dc-86b7-93fbb6f7acd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3630aa55e83f4c70a6780d5f7414374a'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:45.489190 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-dfe1751a-3c5a-48dc-86b7-93fbb6f7acd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3630aa55e83f4c70a6780d5f7414374a'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:45.490109 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-dfe1751a-3c5a-48dc-86b7-93fbb6f7acd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3630aa55e83f4c70a6780d5f7414374a'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:45.490803 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-dfe1751a-3c5a-48dc-86b7-93fbb6f7acd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3630aa55e83f4c70a6780d5f7414374a'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:45.506813 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1115/2198] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:42:45 2022] DELETE /identity/v3/users/3630aa55e83f4c70a6780d5f7414374a => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:45.512980 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96f8ae-de8e-414f-8d81-72c14b1e7429 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:45.513466 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96f8ae-de8e-414f-8d81-72c14b1e7429 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:45.513466 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96f8ae-de8e-414f-8d81-72c14b1e7429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:45.540510 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-cb96f8ae-de8e-414f-8d81-72c14b1e7429 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:45.541939 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-cb96f8ae-de8e-414f-8d81-72c14b1e7429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:45.549652 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe1751a-3c5a-48dc-86b7-93fbb6f7acd5 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:45.551314 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-dfe1751a-3c5a-48dc-86b7-93fbb6f7acd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:45.628317 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1084/2199] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:42:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:45.639450 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96f8ae-de8e-414f-8d81-72c14b1e7429 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:45.640707 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-cb96f8ae-de8e-414f-8d81-72c14b1e7429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:45.646871 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe1751a-3c5a-48dc-86b7-93fbb6f7acd5 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:45.652996 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd4b153-cbfe-4452-8085-ad01edc982a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:45.654278 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4b153-cbfe-4452-8085-ad01edc982a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:45.654637 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4b153-cbfe-4452-8085-ad01edc982a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:45.654986 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4b153-cbfe-4452-8085-ad01edc982a9 admin admin] PATH_INFO: `/v3/users/1672e4a297e943b6ad5752e9271a9fdf` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:45.656734 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd4b153-cbfe-4452-8085-ad01edc982a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1672e4a297e943b6ad5752e9271a9fdf)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:45.663341 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd4b153-cbfe-4452-8085-ad01edc982a9 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:45.700654 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:45.731639 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96f8ae-de8e-414f-8d81-72c14b1e7429 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:45.736493 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045e5600-ae68-48f4-86f1-b6931d031948 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:45.738254 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045e5600-ae68-48f4-86f1-b6931d031948 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:45.738782 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045e5600-ae68-48f4-86f1-b6931d031948 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:45.739201 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045e5600-ae68-48f4-86f1-b6931d031948 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:45.739864 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-5fd4b153-cbfe-4452-8085-ad01edc982a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1672e4a297e943b6ad5752e9271a9fdf'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:45.756541 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-5fd4b153-cbfe-4452-8085-ad01edc982a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1672e4a297e943b6ad5752e9271a9fdf'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:45.757901 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045e5600-ae68-48f4-86f1-b6931d031948 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:45.759588 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045e5600-ae68-48f4-86f1-b6931d031948 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:45.769424 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-5fd4b153-cbfe-4452-8085-ad01edc982a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1672e4a297e943b6ad5752e9271a9fdf'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:45.770324 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-5fd4b153-cbfe-4452-8085-ad01edc982a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1672e4a297e943b6ad5752e9271a9fdf'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:45.773520 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1085/2200] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:42:45.777895 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-5fd4b153-cbfe-4452-8085-ad01edc982a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1672e4a297e943b6ad5752e9271a9fdf'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:45.790368 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1116/2201] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:42:45 2022] DELETE /identity/v3/users/1672e4a297e943b6ad5752e9271a9fdf => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:45.795558 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537cdb0-3168-4b8c-baa5-c7f0ae290915 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:45.795925 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537cdb0-3168-4b8c-baa5-c7f0ae290915 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:45.796274 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537cdb0-3168-4b8c-baa5-c7f0ae290915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:45.820450 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-1537cdb0-3168-4b8c-baa5-c7f0ae290915 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:45.821756 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-1537cdb0-3168-4b8c-baa5-c7f0ae290915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:45.901628 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1086/2202] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:42:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:45.914297 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd4b153-cbfe-4452-8085-ad01edc982a9 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:45.915919 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-5fd4b153-cbfe-4452-8085-ad01edc982a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:46.027441 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd4b153-cbfe-4452-8085-ad01edc982a9 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:46.032125 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0109e206-b93d-4f13-9a8b-9186d23e8335 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:46.033714 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109e206-b93d-4f13-9a8b-9186d23e8335 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:46.034142 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109e206-b93d-4f13-9a8b-9186d23e8335 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:46.034540 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109e206-b93d-4f13-9a8b-9186d23e8335 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:46.047196 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0109e206-b93d-4f13-9a8b-9186d23e8335 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:46.049852 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0109e206-b93d-4f13-9a8b-9186d23e8335 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:46.061747 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1117/2203] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:42:46.227021 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1537cdb0-3168-4b8c-baa5-c7f0ae290915 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:46.228930 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-1537cdb0-3168-4b8c-baa5-c7f0ae290915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:46.255211 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1537cdb0-3168-4b8c-baa5-c7f0ae290915 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:46.270689 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701c6ae6-7865-4f12-9c47-a567736240f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:46.271575 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c6ae6-7865-4f12-9c47-a567736240f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:46.271857 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c6ae6-7865-4f12-9c47-a567736240f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:46.272131 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c6ae6-7865-4f12-9c47-a567736240f1 admin admin] PATH_INFO: `/v3/projects/1431292bdb224b098c38d8dbf641d9a3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:46.273368 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701c6ae6-7865-4f12-9c47-a567736240f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1431292bdb224b098c38d8dbf641d9a3)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:46.275362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701c6ae6-7865-4f12-9c47-a567736240f1 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:46.320938 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:46.330033 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-701c6ae6-7865-4f12-9c47-a567736240f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1431292bdb224b098c38d8dbf641d9a3', 'request_id': 'req-701c6ae6-7865-4f12-9c47-a567736240f1'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:46.336783 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-701c6ae6-7865-4f12-9c47-a567736240f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1431292bdb224b098c38d8dbf641d9a3', 'request_id': 'req-701c6ae6-7865-4f12-9c47-a567736240f1'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:46.347939 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-701c6ae6-7865-4f12-9c47-a567736240f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1431292bdb224b098c38d8dbf641d9a3', 'request_id': 'req-701c6ae6-7865-4f12-9c47-a567736240f1'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:46.366745 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-701c6ae6-7865-4f12-9c47-a567736240f1 admin admin] The token cache is being invalidate because project 1431292bdb224b098c38d8dbf641d9a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:42:46.367038 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-701c6ae6-7865-4f12-9c47-a567736240f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:46.370043 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-701c6ae6-7865-4f12-9c47-a567736240f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:46.373662 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1087/2204] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:42:46 2022] DELETE /identity/v3/projects/1431292bdb224b098c38d8dbf641d9a3 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:42:46.379271 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d6e1dc-64eb-48b8-81ce-6bfdb1d07304 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:46.379701 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d6e1dc-64eb-48b8-81ce-6bfdb1d07304 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:46.380798 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d6e1dc-64eb-48b8-81ce-6bfdb1d07304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:46.402342 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-17d6e1dc-64eb-48b8-81ce-6bfdb1d07304 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:46.403745 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-17d6e1dc-64eb-48b8-81ce-6bfdb1d07304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:46.487277 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1118/2205] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:42:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:46.499259 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf908d4-a236-45e1-852c-d3bd38fa994d None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:46.499556 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf908d4-a236-45e1-852c-d3bd38fa994d None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:46.499838 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf908d4-a236-45e1-852c-d3bd38fa994d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:46.506098 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d6e1dc-64eb-48b8-81ce-6bfdb1d07304 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:46.508448 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-17d6e1dc-64eb-48b8-81ce-6bfdb1d07304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:46.524489 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d6e1dc-64eb-48b8-81ce-6bfdb1d07304 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:46.528140 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-caf908d4-a236-45e1-852c-d3bd38fa994d None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:46.529375 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-caf908d4-a236-45e1-852c-d3bd38fa994d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:46.530268 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cb1256-2ced-4e26-900f-ae24c4416f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:46.531232 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb1256-2ced-4e26-900f-ae24c4416f5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:46.531549 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb1256-2ced-4e26-900f-ae24c4416f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:46.531801 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb1256-2ced-4e26-900f-ae24c4416f5d admin admin] PATH_INFO: `/v3/projects/8b9ff09d98a6448f97bacf0da45b4510` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:46.534367 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cb1256-2ced-4e26-900f-ae24c4416f5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b9ff09d98a6448f97bacf0da45b4510)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:46.536340 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cb1256-2ced-4e26-900f-ae24c4416f5d admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:46.538870 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1088/2206] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:42:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:46.542946 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf908d4-a236-45e1-852c-d3bd38fa994d None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:46.553134 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf908d4-a236-45e1-852c-d3bd38fa994d None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:46.558060 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c4366a-062f-4844-94cb-dbbd5ce8d084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:46.558861 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c4366a-062f-4844-94cb-dbbd5ce8d084 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:46.559201 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c4366a-062f-4844-94cb-dbbd5ce8d084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:46.559698 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c4366a-062f-4844-94cb-dbbd5ce8d084 admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:46.560500 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c4366a-062f-4844-94cb-dbbd5ce8d084 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:42:46.560817 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c4366a-062f-4844-94cb-dbbd5ce8d084 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:46.562531 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c4366a-062f-4844-94cb-dbbd5ce8d084 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:46.575313 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1089/2207] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:42:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:42:46.578891 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:46.580922 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c4366a-062f-4844-94cb-dbbd5ce8d084 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:46.594573 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a3cb1256-2ced-4e26-900f-ae24c4416f5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b9ff09d98a6448f97bacf0da45b4510', 'request_id': 'req-a3cb1256-2ced-4e26-900f-ae24c4416f5d'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:46.595376 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c4366a-062f-4844-94cb-dbbd5ce8d084 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:46.601938 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a3cb1256-2ced-4e26-900f-ae24c4416f5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b9ff09d98a6448f97bacf0da45b4510', 'request_id': 'req-a3cb1256-2ced-4e26-900f-ae24c4416f5d'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:46.607775 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a3cb1256-2ced-4e26-900f-ae24c4416f5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b9ff09d98a6448f97bacf0da45b4510', 'request_id': 'req-a3cb1256-2ced-4e26-900f-ae24c4416f5d'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:46.620930 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a3cb1256-2ced-4e26-900f-ae24c4416f5d admin admin] The token cache is being invalidate because project 8b9ff09d98a6448f97bacf0da45b4510 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:42:46.621220 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a3cb1256-2ced-4e26-900f-ae24c4416f5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:46.621952 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a3cb1256-2ced-4e26-900f-ae24c4416f5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:46.623792 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1119/2208] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:42:46 2022] DELETE /identity/v3/projects/8b9ff09d98a6448f97bacf0da45b4510 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:46.628325 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cb1256-2ced-4e26-900f-ae24c4416f5d admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:46.630158 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-a3cb1256-2ced-4e26-900f-ae24c4416f5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:46.658529 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb9ebb9-fbec-4c10-8d3e-eeda3f3da1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:46.659392 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb9ebb9-fbec-4c10-8d3e-eeda3f3da1d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:46.659682 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb9ebb9-fbec-4c10-8d3e-eeda3f3da1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:46.659957 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb9ebb9-fbec-4c10-8d3e-eeda3f3da1d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:46.660643 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb9ebb9-fbec-4c10-8d3e-eeda3f3da1d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:46.662595 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb9ebb9-fbec-4c10-8d3e-eeda3f3da1d7 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:46.696943 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1090/2209] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:42:46 2022] POST /identity/v3/projects => generated 379 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:46.701015 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb9ebb9-fbec-4c10-8d3e-eeda3f3da1d7 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:46.702087 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cb1256-2ced-4e26-900f-ae24c4416f5d admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:46.703374 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-dcb9ebb9-fbec-4c10-8d3e-eeda3f3da1d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:46.766097 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3268f16a-bff3-4d15-862d-dc1c0ffe1bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:46.767763 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3268f16a-bff3-4d15-862d-dc1c0ffe1bd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:46.768057 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3268f16a-bff3-4d15-862d-dc1c0ffe1bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:46.768429 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3268f16a-bff3-4d15-862d-dc1c0ffe1bd6 admin admin] PATH_INFO: `/v3/users/59bbe0c75e6b4b969eecc1d50fc11965` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:46.770128 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3268f16a-bff3-4d15-862d-dc1c0ffe1bd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59bbe0c75e6b4b969eecc1d50fc11965)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:46.772216 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3268f16a-bff3-4d15-862d-dc1c0ffe1bd6 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:46.775170 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb9ebb9-fbec-4c10-8d3e-eeda3f3da1d7 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:46.781350 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c614832-f1f4-479e-bc5a-99ae45b029f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:46.782254 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c614832-f1f4-479e-bc5a-99ae45b029f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:46.783514 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c614832-f1f4-479e-bc5a-99ae45b029f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:46.783795 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c614832-f1f4-479e-bc5a-99ae45b029f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:46.784527 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c614832-f1f4-479e-bc5a-99ae45b029f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:46.786550 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c614832-f1f4-479e-bc5a-99ae45b029f9 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:46.808121 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1091/2210] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:42:46 2022] POST /identity/v3/users => generated 290 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:42:46.812475 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c614832-f1f4-479e-bc5a-99ae45b029f9 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:46.824644 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c614832-f1f4-479e-bc5a-99ae45b029f9 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:46.830426 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c398d-9bf0-4349-8aac-bc5ce7845a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:46.830711 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-3268f16a-bff3-4d15-862d-dc1c0ffe1bd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59bbe0c75e6b4b969eecc1d50fc11965'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:46.831098 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c398d-9bf0-4349-8aac-bc5ce7845a4b admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:46.831384 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c398d-9bf0-4349-8aac-bc5ce7845a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:46.831663 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c398d-9bf0-4349-8aac-bc5ce7845a4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:46.832389 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602c398d-9bf0-4349-8aac-bc5ce7845a4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:46.834046 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602c398d-9bf0-4349-8aac-bc5ce7845a4b admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:46.842869 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-3268f16a-bff3-4d15-862d-dc1c0ffe1bd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59bbe0c75e6b4b969eecc1d50fc11965'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:46.845439 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1092/2211] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:46.849751 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c398d-9bf0-4349-8aac-bc5ce7845a4b admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:46.855104 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-3268f16a-bff3-4d15-862d-dc1c0ffe1bd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59bbe0c75e6b4b969eecc1d50fc11965'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:46.855740 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-3268f16a-bff3-4d15-862d-dc1c0ffe1bd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59bbe0c75e6b4b969eecc1d50fc11965'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:46.856351 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-3268f16a-bff3-4d15-862d-dc1c0ffe1bd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59bbe0c75e6b4b969eecc1d50fc11965'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:46.858776 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:46.860178 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c398d-9bf0-4349-8aac-bc5ce7845a4b admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:46.865209 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac98798-0851-40f7-9faf-8df3dd596e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:46.865901 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac98798-0851-40f7-9faf-8df3dd596e67 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:46.866190 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac98798-0851-40f7-9faf-8df3dd596e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:46.866466 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac98798-0851-40f7-9faf-8df3dd596e67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:46.867271 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac98798-0851-40f7-9faf-8df3dd596e67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:46.868878 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac98798-0851-40f7-9faf-8df3dd596e67 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:46.873678 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1120/2212] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:42:46 2022] DELETE /identity/v3/users/59bbe0c75e6b4b969eecc1d50fc11965 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:42:46.874946 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:46.885104 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1093/2213] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:42:46.886502 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3268f16a-bff3-4d15-862d-dc1c0ffe1bd6 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:46.888110 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-3268f16a-bff3-4d15-862d-dc1c0ffe1bd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:46.964201 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3268f16a-bff3-4d15-862d-dc1c0ffe1bd6 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:46.968853 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c887e646-7779-4262-9236-15f2cb56e618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:46.969786 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887e646-7779-4262-9236-15f2cb56e618 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:46.970222 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887e646-7779-4262-9236-15f2cb56e618 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:46.970595 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887e646-7779-4262-9236-15f2cb56e618 admin admin] PATH_INFO: `/v3/projects/d0b95979aebe47bc92c387de59394942/users/bf905723382e4372ae63d77a376982ff/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:46.988868 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c887e646-7779-4262-9236-15f2cb56e618 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0b95979aebe47bc92c387de59394942, user_id=bf905723382e4372ae63d77a376982ff, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:46.991661 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c887e646-7779-4262-9236-15f2cb56e618 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:47.010531 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1121/2214] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:46 2022] PUT /identity/v3/projects/d0b95979aebe47bc92c387de59394942/users/bf905723382e4372ae63d77a376982ff/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:47.012375 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59220487-46c2-4538-83b2-f689fb9ced79 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:47.012744 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59220487-46c2-4538-83b2-f689fb9ced79 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:47.013176 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59220487-46c2-4538-83b2-f689fb9ced79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:47.036514 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-59220487-46c2-4538-83b2-f689fb9ced79 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:47.036514 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-59220487-46c2-4538-83b2-f689fb9ced79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:47.109748 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1094/2215] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:42:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:47.120480 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c887e646-7779-4262-9236-15f2cb56e618 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:47.122328 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-c887e646-7779-4262-9236-15f2cb56e618 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:47.175666 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59220487-46c2-4538-83b2-f689fb9ced79 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:47.177634 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-59220487-46c2-4538-83b2-f689fb9ced79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:47.196973 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59220487-46c2-4538-83b2-f689fb9ced79 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:47.197502 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c887e646-7779-4262-9236-15f2cb56e618 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:47.201303 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a758d4db-6eae-4362-837b-dada527da004 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:47.201903 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a758d4db-6eae-4362-837b-dada527da004 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:47.202191 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a758d4db-6eae-4362-837b-dada527da004 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:47.202563 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a758d4db-6eae-4362-837b-dada527da004 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:47.203732 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78c540c-216f-4e7e-aa88-66593facc3e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:47.205985 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c540c-216f-4e7e-aa88-66593facc3e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:47.206352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c540c-216f-4e7e-aa88-66593facc3e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:47.206704 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c540c-216f-4e7e-aa88-66593facc3e4 admin admin] PATH_INFO: `/v3/projects/499728cb599a46e49b3966880476c243` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:47.208322 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78c540c-216f-4e7e-aa88-66593facc3e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=499728cb599a46e49b3966880476c243)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:47.212760 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78c540c-216f-4e7e-aa88-66593facc3e4 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:47.213492 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a758d4db-6eae-4362-837b-dada527da004 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:47.215548 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a758d4db-6eae-4362-837b-dada527da004 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:47.231451 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1122/2216] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:42:47.292181 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-a78c540c-216f-4e7e-aa88-66593facc3e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '499728cb599a46e49b3966880476c243', 'request_id': 'req-a78c540c-216f-4e7e-aa88-66593facc3e4'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:47.300181 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-a78c540c-216f-4e7e-aa88-66593facc3e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '499728cb599a46e49b3966880476c243', 'request_id': 'req-a78c540c-216f-4e7e-aa88-66593facc3e4'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:47.306607 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-a78c540c-216f-4e7e-aa88-66593facc3e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '499728cb599a46e49b3966880476c243', 'request_id': 'req-a78c540c-216f-4e7e-aa88-66593facc3e4'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:47.321968 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-a78c540c-216f-4e7e-aa88-66593facc3e4 admin admin] The token cache is being invalidate because project 499728cb599a46e49b3966880476c243 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:42:47.322261 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-a78c540c-216f-4e7e-aa88-66593facc3e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:47.322948 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-a78c540c-216f-4e7e-aa88-66593facc3e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:47.325194 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1095/2217] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:42:47 2022] DELETE /identity/v3/projects/499728cb599a46e49b3966880476c243 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:42:47.330560 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1202da58-7f27-40f2-a3ed-c4785e23692b None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:47.330925 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1202da58-7f27-40f2-a3ed-c4785e23692b None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:47.331694 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1202da58-7f27-40f2-a3ed-c4785e23692b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:47.356707 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-1202da58-7f27-40f2-a3ed-c4785e23692b None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:47.357972 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-1202da58-7f27-40f2-a3ed-c4785e23692b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:47.429844 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1123/2218] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:42:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:47.437798 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b826951-b3dc-4161-b7f4-846889fbd7be None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:47.438179 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b826951-b3dc-4161-b7f4-846889fbd7be None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:47.438518 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b826951-b3dc-4161-b7f4-846889fbd7be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:47.466039 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-6b826951-b3dc-4161-b7f4-846889fbd7be None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:47.467212 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-6b826951-b3dc-4161-b7f4-846889fbd7be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:47.476729 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1096/2219] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:42:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:47.480637 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1202da58-7f27-40f2-a3ed-c4785e23692b None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:47.490657 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1202da58-7f27-40f2-a3ed-c4785e23692b None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:47.495644 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c31924f-a6da-494f-bb34-c8b975a73203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:47.496315 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c31924f-a6da-494f-bb34-c8b975a73203 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:47.496604 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c31924f-a6da-494f-bb34-c8b975a73203 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:47.496876 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c31924f-a6da-494f-bb34-c8b975a73203 admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:47.497649 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c31924f-a6da-494f-bb34-c8b975a73203 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:42:47.499875 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c31924f-a6da-494f-bb34-c8b975a73203 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:47.499875 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c31924f-a6da-494f-bb34-c8b975a73203 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:47.521263 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1124/2220] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:42:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:42:47.522576 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b826951-b3dc-4161-b7f4-846889fbd7be None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:47.538713 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b826951-b3dc-4161-b7f4-846889fbd7be None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:47.545557 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e2f114-1285-442d-a827-648111b5df83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:47.546277 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e2f114-1285-442d-a827-648111b5df83 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:47.546569 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e2f114-1285-442d-a827-648111b5df83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:47.547153 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e2f114-1285-442d-a827-648111b5df83 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:47.547914 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e2f114-1285-442d-a827-648111b5df83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:47.550656 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e2f114-1285-442d-a827-648111b5df83 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:47.581201 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1097/2221] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:42:47 2022] POST /identity/v3/projects => generated 369 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:47.585457 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c31924f-a6da-494f-bb34-c8b975a73203 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:47.592033 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:47.597862 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c31924f-a6da-494f-bb34-c8b975a73203 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:47.665507 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc856439-8532-4e03-b108-e6ef8408030d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:47.666397 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc856439-8532-4e03-b108-e6ef8408030d admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:47.666684 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc856439-8532-4e03-b108-e6ef8408030d admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:47.666966 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc856439-8532-4e03-b108-e6ef8408030d admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:47.667692 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc856439-8532-4e03-b108-e6ef8408030d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:47.670319 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc856439-8532-4e03-b108-e6ef8408030d admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:47.701676 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1125/2222] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:42:47 2022] POST /identity/v3/users => generated 285 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:42:47.706472 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e2f114-1285-442d-a827-648111b5df83 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:47.718077 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e2f114-1285-442d-a827-648111b5df83 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:47.723952 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406e381d-a351-4e1b-8da6-fb16707ced10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:47.724631 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e381d-a351-4e1b-8da6-fb16707ced10 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:47.726410 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e381d-a351-4e1b-8da6-fb16707ced10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:47.726410 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e381d-a351-4e1b-8da6-fb16707ced10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:47.726410 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406e381d-a351-4e1b-8da6-fb16707ced10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:47.743033 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406e381d-a351-4e1b-8da6-fb16707ced10 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:47.765521 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1098/2223] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:47 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:47.770016 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc856439-8532-4e03-b108-e6ef8408030d admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:47.795035 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc856439-8532-4e03-b108-e6ef8408030d admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:47.795874 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dde1f7-6946-46ec-9a5b-34a1e0c1efba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:47.796466 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dde1f7-6946-46ec-9a5b-34a1e0c1efba admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:47.797158 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dde1f7-6946-46ec-9a5b-34a1e0c1efba admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:47.797604 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dde1f7-6946-46ec-9a5b-34a1e0c1efba admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:47.799603 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dde1f7-6946-46ec-9a5b-34a1e0c1efba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:47.801254 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dde1f7-6946-46ec-9a5b-34a1e0c1efba admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:47.813774 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1126/2224] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:47 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:47.817538 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406e381d-a351-4e1b-8da6-fb16707ced10 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:47.828376 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406e381d-a351-4e1b-8da6-fb16707ced10 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:47.834253 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541bd302-b757-4bf4-9cf7-1ab195ac808f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:47.835188 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541bd302-b757-4bf4-9cf7-1ab195ac808f admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:47.835475 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541bd302-b757-4bf4-9cf7-1ab195ac808f admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:47.835747 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541bd302-b757-4bf4-9cf7-1ab195ac808f admin admin] PATH_INFO: `/v3/projects/51279c2d02e649098b1b6c6cce948c6f/users/c14135ea96a641018bf6b8d9c4506094/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:47.856479 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541bd302-b757-4bf4-9cf7-1ab195ac808f admin admin] RBAC: Authorizing `identity:create_grant(project_id=51279c2d02e649098b1b6c6cce948c6f, user_id=c14135ea96a641018bf6b8d9c4506094, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:47.859882 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541bd302-b757-4bf4-9cf7-1ab195ac808f admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:47.878109 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1099/2225] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:47 2022] PUT /identity/v3/projects/51279c2d02e649098b1b6c6cce948c6f/users/c14135ea96a641018bf6b8d9c4506094/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:47.888711 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339334ec-914a-427b-a092-0b94e6df8721 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:47.888997 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339334ec-914a-427b-a092-0b94e6df8721 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:47.889366 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339334ec-914a-427b-a092-0b94e6df8721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:47.917027 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-339334ec-914a-427b-a092-0b94e6df8721 None None] MFA Rules not processed for user `c14135ea96a641018bf6b8d9c4506094`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:47.918399 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-339334ec-914a-427b-a092-0b94e6df8721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:47.995995 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1127/2226] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:42:47 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:48.001853 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541bd302-b757-4bf4-9cf7-1ab195ac808f admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:48.036024 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:48.110475 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541bd302-b757-4bf4-9cf7-1ab195ac808f admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:48.116743 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9841f3b6-bb18-4a44-85b5-19e84ba71cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:48.117464 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841f3b6-bb18-4a44-85b5-19e84ba71cf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:48.117781 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841f3b6-bb18-4a44-85b5-19e84ba71cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:48.118057 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841f3b6-bb18-4a44-85b5-19e84ba71cf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:48.118770 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9841f3b6-bb18-4a44-85b5-19e84ba71cf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:48.120705 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9841f3b6-bb18-4a44-85b5-19e84ba71cf7 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:48.158203 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1100/2227] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:42:48 2022] POST /identity/v3/projects => generated 369 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:48.162734 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339334ec-914a-427b-a092-0b94e6df8721 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:48.174892 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339334ec-914a-427b-a092-0b94e6df8721 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:48.256707 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db88c506-28b4-424b-bf05-65c40951fcb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:48.257445 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db88c506-28b4-424b-bf05-65c40951fcb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:48.257762 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db88c506-28b4-424b-bf05-65c40951fcb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:48.258074 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db88c506-28b4-424b-bf05-65c40951fcb7 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:48.258794 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db88c506-28b4-424b-bf05-65c40951fcb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:48.260735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db88c506-28b4-424b-bf05-65c40951fcb7 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:48.288760 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1128/2228] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:42:48 2022] POST /identity/v3/users => generated 291 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:48.290432 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9841f3b6-bb18-4a44-85b5-19e84ba71cf7 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:48.313235 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9841f3b6-bb18-4a44-85b5-19e84ba71cf7 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:48.319809 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9853f2d5-6204-4ed7-9560-4c5faba611e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:48.320780 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9853f2d5-6204-4ed7-9560-4c5faba611e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:48.321077 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9853f2d5-6204-4ed7-9560-4c5faba611e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:48.321354 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9853f2d5-6204-4ed7-9560-4c5faba611e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:48.322140 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9853f2d5-6204-4ed7-9560-4c5faba611e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:48.323761 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9853f2d5-6204-4ed7-9560-4c5faba611e3 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:48.336435 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1101/2229] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:48 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:48.339995 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db88c506-28b4-424b-bf05-65c40951fcb7 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:48.346655 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:48.353367 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db88c506-28b4-424b-bf05-65c40951fcb7 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:48.362285 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb048f1-c5f4-4613-a7b2-724eb6086009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:48.366867 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb048f1-c5f4-4613-a7b2-724eb6086009 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:48.366867 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb048f1-c5f4-4613-a7b2-724eb6086009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:48.366867 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb048f1-c5f4-4613-a7b2-724eb6086009 admin admin] PATH_INFO: `/v3/projects/b0bd9878ee6f45d6950d7f8e2e3effae/users/2cdc01da6b004de490ded3539b8ff293/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:48.385490 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb048f1-c5f4-4613-a7b2-724eb6086009 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0bd9878ee6f45d6950d7f8e2e3effae, user_id=2cdc01da6b004de490ded3539b8ff293, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:48.388570 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb048f1-c5f4-4613-a7b2-724eb6086009 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:48.404043 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1129/2230] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:48 2022] PUT /identity/v3/projects/b0bd9878ee6f45d6950d7f8e2e3effae/users/2cdc01da6b004de490ded3539b8ff293/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:48.408643 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9853f2d5-6204-4ed7-9560-4c5faba611e3 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:48.421124 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9853f2d5-6204-4ed7-9560-4c5faba611e3 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:48.526539 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28c40da-e8de-4bb7-a78b-4aa114196ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:48.527941 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28c40da-e8de-4bb7-a78b-4aa114196ce3 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:48.528408 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28c40da-e8de-4bb7-a78b-4aa114196ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:48.528715 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28c40da-e8de-4bb7-a78b-4aa114196ce3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:48.529987 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28c40da-e8de-4bb7-a78b-4aa114196ce3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:48.532235 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28c40da-e8de-4bb7-a78b-4aa114196ce3 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:48.556493 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1102/2231] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:48 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:42:48.560204 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb048f1-c5f4-4613-a7b2-724eb6086009 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:48.575392 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb048f1-c5f4-4613-a7b2-724eb6086009 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:48.589479 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0be5bf-c8dc-4305-813d-83662fece0e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:48.590762 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0be5bf-c8dc-4305-813d-83662fece0e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:48.591259 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0be5bf-c8dc-4305-813d-83662fece0e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:48.592041 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0be5bf-c8dc-4305-813d-83662fece0e0 admin admin] PATH_INFO: `/v3/projects/b0bd9878ee6f45d6950d7f8e2e3effae/users/2cdc01da6b004de490ded3539b8ff293/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:48.598261 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0be5bf-c8dc-4305-813d-83662fece0e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0bd9878ee6f45d6950d7f8e2e3effae, user_id=2cdc01da6b004de490ded3539b8ff293, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:48.601999 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0be5bf-c8dc-4305-813d-83662fece0e0 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:48.632908 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1130/2232] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:48 2022] PUT /identity/v3/projects/b0bd9878ee6f45d6950d7f8e2e3effae/users/2cdc01da6b004de490ded3539b8ff293/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:48.649913 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff780e8-e55e-44c5-be62-16807cf74c18 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:48.650207 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff780e8-e55e-44c5-be62-16807cf74c18 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:48.650492 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff780e8-e55e-44c5-be62-16807cf74c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:48.707495 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-5ff780e8-e55e-44c5-be62-16807cf74c18 None None] MFA Rules not processed for user `2cdc01da6b004de490ded3539b8ff293`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:48.709714 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-5ff780e8-e55e-44c5-be62-16807cf74c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:48.853754 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1103/2233] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:42:48 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:42:48.864553 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0be5bf-c8dc-4305-813d-83662fece0e0 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:48.866964 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-da0be5bf-c8dc-4305-813d-83662fece0e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:48.948078 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0be5bf-c8dc-4305-813d-83662fece0e0 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:48.955797 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2344e84-f891-4dbd-8def-165e28a64b39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:48.956604 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2344e84-f891-4dbd-8def-165e28a64b39 service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:48.957192 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2344e84-f891-4dbd-8def-165e28a64b39 service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:48.957866 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2344e84-f891-4dbd-8def-165e28a64b39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:48.977003 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2344e84-f891-4dbd-8def-165e28a64b39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:48.978531 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2344e84-f891-4dbd-8def-165e28a64b39 service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:48.990937 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1131/2234] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:48 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:42:50.371456 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e28b15-9ec6-411f-803d-b09981724c98 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:50.372579 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e28b15-9ec6-411f-803d-b09981724c98 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:50.373336 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e28b15-9ec6-411f-803d-b09981724c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:50.414912 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-09e28b15-9ec6-411f-803d-b09981724c98 None None] MFA Rules not processed for user `85d9c4c57681447c898ebfd7990b05b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:50.416395 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-09e28b15-9ec6-411f-803d-b09981724c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:50.430756 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:50.492997 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1104/2235] 10.222.0.13 () {58 vars in 1009 bytes} [Fri Jun 24 08:42:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:50.571812 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2344e84-f891-4dbd-8def-165e28a64b39 service cinder] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:50.573384 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-f2344e84-f891-4dbd-8def-165e28a64b39 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:50.689023 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2344e84-f891-4dbd-8def-165e28a64b39 service cinder] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:50.696734 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad3ac08-b879-434c-bdaf-8aba3e960506 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec8c2f113cf41a0939b81f38fbee813', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:50.697409 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad3ac08-b879-434c-bdaf-8aba3e960506 service swift] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:50.697732 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad3ac08-b879-434c-bdaf-8aba3e960506 service swift] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:50.698614 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad3ac08-b879-434c-bdaf-8aba3e960506 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:50.716140 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad3ac08-b879-434c-bdaf-8aba3e960506 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:50.717811 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad3ac08-b879-434c-bdaf-8aba3e960506 service swift] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:50.748765 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1132/2236] 10.222.0.13 () {60 vars in 1384 bytes} [Fri Jun 24 08:42:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 08:42:51.359739 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e28b15-9ec6-411f-803d-b09981724c98 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:51.506874 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e28b15-9ec6-411f-803d-b09981724c98 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:51.514570 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d440a27c-cd5a-49b4-94e9-c68d9076f682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:51.515752 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d440a27c-cd5a-49b4-94e9-c68d9076f682 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:51.516202 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d440a27c-cd5a-49b4-94e9-c68d9076f682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:51.516639 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d440a27c-cd5a-49b4-94e9-c68d9076f682 admin admin] PATH_INFO: `/v3/users/c14135ea96a641018bf6b8d9c4506094` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:51.518286 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d440a27c-cd5a-49b4-94e9-c68d9076f682 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c14135ea96a641018bf6b8d9c4506094)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:51.520803 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d440a27c-cd5a-49b4-94e9-c68d9076f682 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:51.620884 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-d440a27c-cd5a-49b4-94e9-c68d9076f682 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c14135ea96a641018bf6b8d9c4506094'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:51.659873 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-d440a27c-cd5a-49b4-94e9-c68d9076f682 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c14135ea96a641018bf6b8d9c4506094'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:51.664297 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-d440a27c-cd5a-49b4-94e9-c68d9076f682 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c14135ea96a641018bf6b8d9c4506094'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:51.680789 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-d440a27c-cd5a-49b4-94e9-c68d9076f682 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c14135ea96a641018bf6b8d9c4506094'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:51.682186 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-d440a27c-cd5a-49b4-94e9-c68d9076f682 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c14135ea96a641018bf6b8d9c4506094'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:51.714271 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1105/2237] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:42:51 2022] DELETE /identity/v3/users/c14135ea96a641018bf6b8d9c4506094 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:42:51.717102 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bfeba7-30af-4677-b265-7f38bf7027b6 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:51.718086 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bfeba7-30af-4677-b265-7f38bf7027b6 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:51.718609 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bfeba7-30af-4677-b265-7f38bf7027b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:51.749838 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-a7bfeba7-30af-4677-b265-7f38bf7027b6 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:51.751469 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-a7bfeba7-30af-4677-b265-7f38bf7027b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:51.857519 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:51.897158 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1133/2238] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:42:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:51.913831 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d440a27c-cd5a-49b4-94e9-c68d9076f682 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:51.916348 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d440a27c-cd5a-49b4-94e9-c68d9076f682 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:51.996634 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:52.021330 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d440a27c-cd5a-49b4-94e9-c68d9076f682 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:52.025948 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715f94da-b74a-4efd-8a58-34aa8ea3068e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:52.026785 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f94da-b74a-4efd-8a58-34aa8ea3068e service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:52.027176 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f94da-b74a-4efd-8a58-34aa8ea3068e service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:52.027558 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f94da-b74a-4efd-8a58-34aa8ea3068e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:52.037972 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715f94da-b74a-4efd-8a58-34aa8ea3068e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:52.039609 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715f94da-b74a-4efd-8a58-34aa8ea3068e service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:52.051265 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1106/2239] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:42:52.592344 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bfeba7-30af-4677-b265-7f38bf7027b6 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:52.594032 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-a7bfeba7-30af-4677-b265-7f38bf7027b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:52.636001 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bfeba7-30af-4677-b265-7f38bf7027b6 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:52.655268 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8913b1-29b6-46e5-bb9b-15dec87f6c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:52.660723 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8913b1-29b6-46e5-bb9b-15dec87f6c35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:52.661806 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8913b1-29b6-46e5-bb9b-15dec87f6c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:52.662565 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8913b1-29b6-46e5-bb9b-15dec87f6c35 admin admin] PATH_INFO: `/v3/projects/51279c2d02e649098b1b6c6cce948c6f` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:52.670901 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8913b1-29b6-46e5-bb9b-15dec87f6c35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51279c2d02e649098b1b6c6cce948c6f)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:52.682269 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8913b1-29b6-46e5-bb9b-15dec87f6c35 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:52.830841 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-6c8913b1-29b6-46e5-bb9b-15dec87f6c35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51279c2d02e649098b1b6c6cce948c6f', 'request_id': 'req-6c8913b1-29b6-46e5-bb9b-15dec87f6c35'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:52.841023 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-6c8913b1-29b6-46e5-bb9b-15dec87f6c35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51279c2d02e649098b1b6c6cce948c6f', 'request_id': 'req-6c8913b1-29b6-46e5-bb9b-15dec87f6c35'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:52.850512 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-6c8913b1-29b6-46e5-bb9b-15dec87f6c35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51279c2d02e649098b1b6c6cce948c6f', 'request_id': 'req-6c8913b1-29b6-46e5-bb9b-15dec87f6c35'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:52.879461 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-6c8913b1-29b6-46e5-bb9b-15dec87f6c35 admin admin] The token cache is being invalidate because project 51279c2d02e649098b1b6c6cce948c6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:42:52.880110 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-6c8913b1-29b6-46e5-bb9b-15dec87f6c35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:52.881000 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-6c8913b1-29b6-46e5-bb9b-15dec87f6c35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:52.883413 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1134/2240] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:42:52 2022] DELETE /identity/v3/projects/51279c2d02e649098b1b6c6cce948c6f => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:52.886985 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715f94da-b74a-4efd-8a58-34aa8ea3068e service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:52.904823 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715f94da-b74a-4efd-8a58-34aa8ea3068e service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:52.999085 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0f9bf7-2aa1-48dc-b241-481218e642ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:53.000173 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f9bf7-2aa1-48dc-b241-481218e642ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:53.000787 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f9bf7-2aa1-48dc-b241-481218e642ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:53.001419 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f9bf7-2aa1-48dc-b241-481218e642ce admin admin] PATH_INFO: `/v3/users/2cdc01da6b004de490ded3539b8ff293` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:53.015758 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0f9bf7-2aa1-48dc-b241-481218e642ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cdc01da6b004de490ded3539b8ff293)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:53.018895 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0f9bf7-2aa1-48dc-b241-481218e642ce admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:53.125792 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7c0f9bf7-2aa1-48dc-b241-481218e642ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cdc01da6b004de490ded3539b8ff293'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:53.162897 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7c0f9bf7-2aa1-48dc-b241-481218e642ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cdc01da6b004de490ded3539b8ff293'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:53.163851 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7c0f9bf7-2aa1-48dc-b241-481218e642ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cdc01da6b004de490ded3539b8ff293'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:53.184335 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7c0f9bf7-2aa1-48dc-b241-481218e642ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cdc01da6b004de490ded3539b8ff293'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:53.186477 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7c0f9bf7-2aa1-48dc-b241-481218e642ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cdc01da6b004de490ded3539b8ff293'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:53.222740 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1107/2241] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:42:52 2022] DELETE /identity/v3/users/2cdc01da6b004de490ded3539b8ff293 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:54.202576 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8913b1-29b6-46e5-bb9b-15dec87f6c35 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:54.205755 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-6c8913b1-29b6-46e5-bb9b-15dec87f6c35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:54.274894 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8913b1-29b6-46e5-bb9b-15dec87f6c35 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:54.483415 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096ced7a-6f1e-400c-9463-ddedea561cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:54.484450 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ced7a-6f1e-400c-9463-ddedea561cb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:54.484751 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ced7a-6f1e-400c-9463-ddedea561cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:54.485126 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ced7a-6f1e-400c-9463-ddedea561cb9 admin admin] PATH_INFO: `/v3/projects/b0bd9878ee6f45d6950d7f8e2e3effae` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:54.487155 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096ced7a-6f1e-400c-9463-ddedea561cb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0bd9878ee6f45d6950d7f8e2e3effae)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:54.489391 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096ced7a-6f1e-400c-9463-ddedea561cb9 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:54.548673 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-096ced7a-6f1e-400c-9463-ddedea561cb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0bd9878ee6f45d6950d7f8e2e3effae', 'request_id': 'req-096ced7a-6f1e-400c-9463-ddedea561cb9'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:54.555620 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-096ced7a-6f1e-400c-9463-ddedea561cb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0bd9878ee6f45d6950d7f8e2e3effae', 'request_id': 'req-096ced7a-6f1e-400c-9463-ddedea561cb9'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:54.561898 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-096ced7a-6f1e-400c-9463-ddedea561cb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0bd9878ee6f45d6950d7f8e2e3effae', 'request_id': 'req-096ced7a-6f1e-400c-9463-ddedea561cb9'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:54.587845 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-096ced7a-6f1e-400c-9463-ddedea561cb9 admin admin] The token cache is being invalidate because project b0bd9878ee6f45d6950d7f8e2e3effae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:42:54.588198 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-096ced7a-6f1e-400c-9463-ddedea561cb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:54.589063 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-096ced7a-6f1e-400c-9463-ddedea561cb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:54.591475 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1135/2242] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:42:54 2022] DELETE /identity/v3/projects/b0bd9878ee6f45d6950d7f8e2e3effae => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:54.596544 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dca57f-1b12-4887-b61b-e7f46dc44fb8 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:54.597084 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dca57f-1b12-4887-b61b-e7f46dc44fb8 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:54.597084 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dca57f-1b12-4887-b61b-e7f46dc44fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:54.631380 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-23dca57f-1b12-4887-b61b-e7f46dc44fb8 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:54.632660 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-23dca57f-1b12-4887-b61b-e7f46dc44fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:54.768717 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1108/2243] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:42:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:54.771042 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d6d0d6-a495-4655-8107-419d622ac441 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:54.771331 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d6d0d6-a495-4655-8107-419d622ac441 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:54.771610 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d6d0d6-a495-4655-8107-419d622ac441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:54.825764 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-62d6d0d6-a495-4655-8107-419d622ac441 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:54.827026 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-62d6d0d6-a495-4655-8107-419d622ac441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:54.854658 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1136/2244] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:42:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:54.855940 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dca57f-1b12-4887-b61b-e7f46dc44fb8 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:54.863186 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:54.873612 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dca57f-1b12-4887-b61b-e7f46dc44fb8 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:54.902629 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4d2d6e-f4fa-4cee-b558-6afee3641773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:54.903673 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4d2d6e-f4fa-4cee-b558-6afee3641773 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:54.904146 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4d2d6e-f4fa-4cee-b558-6afee3641773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:54.905241 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4d2d6e-f4fa-4cee-b558-6afee3641773 admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:54.906064 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4d2d6e-f4fa-4cee-b558-6afee3641773 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:42:54.906384 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4d2d6e-f4fa-4cee-b558-6afee3641773 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:54.910264 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4d2d6e-f4fa-4cee-b558-6afee3641773 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:54.931143 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1109/2245] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:42:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:42:54.936085 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d6d0d6-a495-4655-8107-419d622ac441 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:54.948764 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:54.954565 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d6d0d6-a495-4655-8107-419d622ac441 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:54.973792 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a26331-759f-4dde-af40-abcef71b3ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:54.975194 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a26331-759f-4dde-af40-abcef71b3ca3 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:54.975671 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a26331-759f-4dde-af40-abcef71b3ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:54.976130 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a26331-759f-4dde-af40-abcef71b3ca3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:54.989552 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a26331-759f-4dde-af40-abcef71b3ca3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:54.991685 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a26331-759f-4dde-af40-abcef71b3ca3 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:55.039176 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1137/2246] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:42:54 2022] POST /identity/v3/projects => generated 379 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:55.043114 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4d2d6e-f4fa-4cee-b558-6afee3641773 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:55.055772 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4d2d6e-f4fa-4cee-b558-6afee3641773 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:55.156696 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8853d1-18a2-4553-a5fb-dd4437a0d398 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:55.156957 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8853d1-18a2-4553-a5fb-dd4437a0d398 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:55.157312 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8853d1-18a2-4553-a5fb-dd4437a0d398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:55.169359 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe433db4-ad13-4fbb-952d-7b4685282b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:55.169359 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe433db4-ad13-4fbb-952d-7b4685282b6d admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:55.169359 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe433db4-ad13-4fbb-952d-7b4685282b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:55.169359 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe433db4-ad13-4fbb-952d-7b4685282b6d admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:55.170238 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe433db4-ad13-4fbb-952d-7b4685282b6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:55.172287 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe433db4-ad13-4fbb-952d-7b4685282b6d admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:55.201166 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-7a8853d1-18a2-4553-a5fb-dd4437a0d398 None None] MFA Rules not processed for user `bf905723382e4372ae63d77a376982ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:55.202516 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-7a8853d1-18a2-4553-a5fb-dd4437a0d398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:55.205995 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1110/2247] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:42:55 2022] POST /identity/v3/users => generated 290 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:42:55.210513 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe433db4-ad13-4fbb-952d-7b4685282b6d admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:55.224247 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe433db4-ad13-4fbb-952d-7b4685282b6d admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:55.230162 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30cac27-4804-421b-b36f-aef1250971a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:55.231692 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30cac27-4804-421b-b36f-aef1250971a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:55.232042 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30cac27-4804-421b-b36f-aef1250971a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:55.232405 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30cac27-4804-421b-b36f-aef1250971a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:55.233209 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30cac27-4804-421b-b36f-aef1250971a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:55.234955 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30cac27-4804-421b-b36f-aef1250971a1 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:55.252917 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1111/2248] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:55 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:42:55.256956 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30cac27-4804-421b-b36f-aef1250971a1 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:55.268809 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30cac27-4804-421b-b36f-aef1250971a1 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:55.274254 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3b800a-b220-4caa-8e60-5e84125a2279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:55.274933 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b800a-b220-4caa-8e60-5e84125a2279 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:55.275223 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b800a-b220-4caa-8e60-5e84125a2279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:55.275502 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b800a-b220-4caa-8e60-5e84125a2279 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:55.276270 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3b800a-b220-4caa-8e60-5e84125a2279 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:55.278042 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3b800a-b220-4caa-8e60-5e84125a2279 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:55.305629 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1138/2250] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:42:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:42:55.310327 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1112/2249] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:55 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:55.311406 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3b800a-b220-4caa-8e60-5e84125a2279 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:55.311678 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8853d1-18a2-4553-a5fb-dd4437a0d398 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:55.314560 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-7a8853d1-18a2-4553-a5fb-dd4437a0d398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:55.332884 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3b800a-b220-4caa-8e60-5e84125a2279 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:55.334992 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8853d1-18a2-4553-a5fb-dd4437a0d398 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:55.343259 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d3f961-adf2-42df-b569-74cecfa7e0e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:55.344209 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d3f961-adf2-42df-b569-74cecfa7e0e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:55.344600 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d3f961-adf2-42df-b569-74cecfa7e0e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:55.344922 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d3f961-adf2-42df-b569-74cecfa7e0e5 admin admin] PATH_INFO: `/v3/projects/b1b493f59cea4eb48989bbaf7904fa7a/users/d62dbf20083d44d28ea49f77a0a00329/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:55.356943 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a70d9fa-5a82-4a7e-9830-4f057418147b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:55.357847 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a70d9fa-5a82-4a7e-9830-4f057418147b admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:55.358142 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a70d9fa-5a82-4a7e-9830-4f057418147b admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:55.358431 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a70d9fa-5a82-4a7e-9830-4f057418147b admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:55.359301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a70d9fa-5a82-4a7e-9830-4f057418147b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:55.362440 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a70d9fa-5a82-4a7e-9830-4f057418147b admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:55.401528 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d3f961-adf2-42df-b569-74cecfa7e0e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1b493f59cea4eb48989bbaf7904fa7a, user_id=d62dbf20083d44d28ea49f77a0a00329, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:55.411867 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d3f961-adf2-42df-b569-74cecfa7e0e5 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:55.439061 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1139/2251] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:42:55 2022] POST /identity/v3/projects => generated 379 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:42:55.439632 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1113/2252] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:55 2022] PUT /identity/v3/projects/b1b493f59cea4eb48989bbaf7904fa7a/users/d62dbf20083d44d28ea49f77a0a00329/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:55.443248 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a70d9fa-5a82-4a7e-9830-4f057418147b admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:55.453733 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac4613-f9b8-4106-8e7a-965df810296e None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:55.454694 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac4613-f9b8-4106-8e7a-965df810296e None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:55.455046 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac4613-f9b8-4106-8e7a-965df810296e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:55.492862 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-70ac4613-f9b8-4106-8e7a-965df810296e None None] MFA Rules not processed for user `d62dbf20083d44d28ea49f77a0a00329`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:55.494844 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-70ac4613-f9b8-4106-8e7a-965df810296e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:55.556929 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a70d9fa-5a82-4a7e-9830-4f057418147b admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:55.562679 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b771ac00-b35c-405e-a491-cd27d0b8ffea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:55.563383 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b771ac00-b35c-405e-a491-cd27d0b8ffea admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:55.564486 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b771ac00-b35c-405e-a491-cd27d0b8ffea admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:55.564595 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b771ac00-b35c-405e-a491-cd27d0b8ffea admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:55.565363 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b771ac00-b35c-405e-a491-cd27d0b8ffea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:55.567543 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b771ac00-b35c-405e-a491-cd27d0b8ffea admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:55.601391 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1140/2253] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:42:55 2022] POST /identity/v3/users => generated 296 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:55.602004 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1114/2254] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:42:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:55.606410 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b771ac00-b35c-405e-a491-cd27d0b8ffea admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:55.607739 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ac4613-f9b8-4106-8e7a-965df810296e None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:55.619921 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b771ac00-b35c-405e-a491-cd27d0b8ffea admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:55.625454 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ac4613-f9b8-4106-8e7a-965df810296e None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:55.625955 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0338639-2a9a-4447-825e-62bd3f7c6003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:55.626987 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0338639-2a9a-4447-825e-62bd3f7c6003 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:55.627409 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0338639-2a9a-4447-825e-62bd3f7c6003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:55.627783 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0338639-2a9a-4447-825e-62bd3f7c6003 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:55.628738 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0338639-2a9a-4447-825e-62bd3f7c6003 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:55.630771 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0338639-2a9a-4447-825e-62bd3f7c6003 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:55.634166 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c8adde-4215-42bf-baca-9dcf18392689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:55.634851 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c8adde-4215-42bf-baca-9dcf18392689 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:55.635041 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c8adde-4215-42bf-baca-9dcf18392689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:55.635324 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c8adde-4215-42bf-baca-9dcf18392689 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:55.636033 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c8adde-4215-42bf-baca-9dcf18392689 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:55.638099 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c8adde-4215-42bf-baca-9dcf18392689 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:55.647294 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1141/2255] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:55 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:55.649140 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0338639-2a9a-4447-825e-62bd3f7c6003 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:55.662825 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0338639-2a9a-4447-825e-62bd3f7c6003 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:55.669265 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193a67a-f397-4ee0-8296-5b648321bead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:55.670345 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193a67a-f397-4ee0-8296-5b648321bead admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:55.670939 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193a67a-f397-4ee0-8296-5b648321bead admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:55.670939 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193a67a-f397-4ee0-8296-5b648321bead admin admin] PATH_INFO: `/v3/projects/81aa67602e4545b0983e9e201551e578/users/ce24dd074ae44e4ea79ccf6451634626/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:55.682289 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:55.695729 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8193a67a-f397-4ee0-8296-5b648321bead admin admin] RBAC: Authorizing `identity:create_grant(project_id=81aa67602e4545b0983e9e201551e578, user_id=ce24dd074ae44e4ea79ccf6451634626, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:55.695969 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1115/2256] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:42:55 2022] POST /identity/v3/projects => generated 377 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:55.696903 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c8adde-4215-42bf-baca-9dcf18392689 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:55.699130 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8193a67a-f397-4ee0-8296-5b648321bead admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:55.719095 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1142/2257] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:55 2022] PUT /identity/v3/projects/81aa67602e4545b0983e9e201551e578/users/ce24dd074ae44e4ea79ccf6451634626/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:55.723534 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193a67a-f397-4ee0-8296-5b648321bead admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:55.729817 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:55.808952 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c8adde-4215-42bf-baca-9dcf18392689 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:55.857790 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193a67a-f397-4ee0-8296-5b648321bead admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:55.878196 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d218b-b06a-4fdf-b146-85ce567e5164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:55.878897 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d218b-b06a-4fdf-b146-85ce567e5164 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:55.881204 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d218b-b06a-4fdf-b146-85ce567e5164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:55.881498 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d218b-b06a-4fdf-b146-85ce567e5164 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:55.882278 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925d218b-b06a-4fdf-b146-85ce567e5164 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:55.889149 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925d218b-b06a-4fdf-b146-85ce567e5164 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:55.944174 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1143/2258] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:55 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:42:55.945682 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d218b-b06a-4fdf-b146-85ce567e5164 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:55.964101 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548dc916-8764-4f19-aea9-1ed91336e13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:55.964798 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548dc916-8764-4f19-aea9-1ed91336e13d admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:55.965100 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548dc916-8764-4f19-aea9-1ed91336e13d admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:55.965382 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548dc916-8764-4f19-aea9-1ed91336e13d admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:55.967890 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548dc916-8764-4f19-aea9-1ed91336e13d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:55.983570 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d218b-b06a-4fdf-b146-85ce567e5164 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:55.991060 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548dc916-8764-4f19-aea9-1ed91336e13d admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:56.011025 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36747d5f-f538-43b1-881d-ef48a311f542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:56.014349 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36747d5f-f538-43b1-881d-ef48a311f542 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:56.014641 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36747d5f-f538-43b1-881d-ef48a311f542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:56.015028 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36747d5f-f538-43b1-881d-ef48a311f542 admin admin] PATH_INFO: `/v3/projects/81aa67602e4545b0983e9e201551e578/users/ce24dd074ae44e4ea79ccf6451634626/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:56.028615 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36747d5f-f538-43b1-881d-ef48a311f542 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81aa67602e4545b0983e9e201551e578, user_id=ce24dd074ae44e4ea79ccf6451634626, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:56.041201 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36747d5f-f538-43b1-881d-ef48a311f542 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:56.049521 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1116/2259] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:42:55 2022] POST /identity/v3/users => generated 289 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:56.054028 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548dc916-8764-4f19-aea9-1ed91336e13d admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:56.076207 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548dc916-8764-4f19-aea9-1ed91336e13d admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:56.084361 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1144/2260] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:55 2022] PUT /identity/v3/projects/81aa67602e4545b0983e9e201551e578/users/ce24dd074ae44e4ea79ccf6451634626/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:56.249889 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2fecd1-1d8c-4ff3-ad91-4d1a0adca389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:56.251703 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2fecd1-1d8c-4ff3-ad91-4d1a0adca389 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:56.253369 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2fecd1-1d8c-4ff3-ad91-4d1a0adca389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:56.254768 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2fecd1-1d8c-4ff3-ad91-4d1a0adca389 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:56.255741 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2fecd1-1d8c-4ff3-ad91-4d1a0adca389 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:56.260281 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2fecd1-1d8c-4ff3-ad91-4d1a0adca389 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:56.318860 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1117/2261] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:56 2022] GET /identity/v3/roles => generated 1420 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:56.328371 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36747d5f-f538-43b1-881d-ef48a311f542 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:56.349407 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36747d5f-f538-43b1-881d-ef48a311f542 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:56.385018 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65197f0e-d323-4906-b380-9831587e3b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:56.386205 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65197f0e-d323-4906-b380-9831587e3b5b admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:56.390257 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65197f0e-d323-4906-b380-9831587e3b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:56.391359 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65197f0e-d323-4906-b380-9831587e3b5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:56.393592 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65197f0e-d323-4906-b380-9831587e3b5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:56.395669 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65197f0e-d323-4906-b380-9831587e3b5b admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:56.504339 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1145/2262] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:56 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:56.513089 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2fecd1-1d8c-4ff3-ad91-4d1a0adca389 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:56.593135 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:56.620326 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2fecd1-1d8c-4ff3-ad91-4d1a0adca389 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:56.633215 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c139a047-52a4-46b9-97a5-e7f82968b26c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:56.634338 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c139a047-52a4-46b9-97a5-e7f82968b26c admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:56.634631 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c139a047-52a4-46b9-97a5-e7f82968b26c admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:56.634933 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c139a047-52a4-46b9-97a5-e7f82968b26c admin admin] PATH_INFO: `/v3/projects/d8f1f250cffa4367ae9a86394d521dcb/users/56d6d56bf14047e393c08e46217c3fd9/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:56.675659 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c139a047-52a4-46b9-97a5-e7f82968b26c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8f1f250cffa4367ae9a86394d521dcb, user_id=56d6d56bf14047e393c08e46217c3fd9, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:56.678984 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c139a047-52a4-46b9-97a5-e7f82968b26c admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:56.744089 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1118/2263] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:56 2022] PUT /identity/v3/projects/d8f1f250cffa4367ae9a86394d521dcb/users/56d6d56bf14047e393c08e46217c3fd9/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:56.745973 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c858d6a-6e31-4eea-b787-88af43aee0d7 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:56.746194 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c858d6a-6e31-4eea-b787-88af43aee0d7 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:56.746442 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c858d6a-6e31-4eea-b787-88af43aee0d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:56.839270 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-4c858d6a-6e31-4eea-b787-88af43aee0d7 None None] MFA Rules not processed for user `56d6d56bf14047e393c08e46217c3fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:56.842069 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-4c858d6a-6e31-4eea-b787-88af43aee0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:57.015327 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1146/2264] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:42:56 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:57.016108 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c139a047-52a4-46b9-97a5-e7f82968b26c admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:57.253152 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c139a047-52a4-46b9-97a5-e7f82968b26c admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:57.265745 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8338e4-debe-4f18-b835-09dc45e16c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:57.268685 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8338e4-debe-4f18-b835-09dc45e16c8d admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:57.268685 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8338e4-debe-4f18-b835-09dc45e16c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:57.268685 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8338e4-debe-4f18-b835-09dc45e16c8d admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:57.268685 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8338e4-debe-4f18-b835-09dc45e16c8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:57.273728 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8338e4-debe-4f18-b835-09dc45e16c8d admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:57.331995 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1119/2265] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:42:57 2022] POST /identity/v3/projects => generated 379 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:57.338258 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c858d6a-6e31-4eea-b787-88af43aee0d7 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:57.363153 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c858d6a-6e31-4eea-b787-88af43aee0d7 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:57.495842 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:57.507958 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee4a4b7-a340-425b-bc5c-01d30ed6c725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:57.508739 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee4a4b7-a340-425b-bc5c-01d30ed6c725 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:57.509402 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee4a4b7-a340-425b-bc5c-01d30ed6c725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:57.509527 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee4a4b7-a340-425b-bc5c-01d30ed6c725 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:57.510640 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee4a4b7-a340-425b-bc5c-01d30ed6c725 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:57.514955 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee4a4b7-a340-425b-bc5c-01d30ed6c725 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:57.566899 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8338e4-debe-4f18-b835-09dc45e16c8d admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:57.568040 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1147/2266] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:42:57 2022] POST /identity/v3/users => generated 296 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:57.592963 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8338e4-debe-4f18-b835-09dc45e16c8d admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:57.631333 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e376084-7bdd-4093-aab1-f3eda27dab45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:57.632062 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e376084-7bdd-4093-aab1-f3eda27dab45 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:57.632282 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e376084-7bdd-4093-aab1-f3eda27dab45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:57.635091 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e376084-7bdd-4093-aab1-f3eda27dab45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:57.636055 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e376084-7bdd-4093-aab1-f3eda27dab45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:57.638668 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e376084-7bdd-4093-aab1-f3eda27dab45 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:57.691139 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee4a4b7-a340-425b-bc5c-01d30ed6c725 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:57.706960 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee4a4b7-a340-425b-bc5c-01d30ed6c725 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:57.711181 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1120/2267] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:57 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:57.724715 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da6790-4e9c-4d21-a0a6-bcc8f36cb5be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:57.730723 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da6790-4e9c-4d21-a0a6-bcc8f36cb5be admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:57.731051 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da6790-4e9c-4d21-a0a6-bcc8f36cb5be admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:57.731340 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da6790-4e9c-4d21-a0a6-bcc8f36cb5be admin admin] PATH_INFO: `/v3/projects/8c7bdfa188d54f53b6fa7780406ab470/users/01edd6421bcf4f99ba676bae307fb597/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:57.818782 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da6790-4e9c-4d21-a0a6-bcc8f36cb5be admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c7bdfa188d54f53b6fa7780406ab470, user_id=01edd6421bcf4f99ba676bae307fb597, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:57.826693 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da6790-4e9c-4d21-a0a6-bcc8f36cb5be admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:57.886006 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1148/2268] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:57 2022] PUT /identity/v3/projects/8c7bdfa188d54f53b6fa7780406ab470/users/01edd6421bcf4f99ba676bae307fb597/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:57.887586 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e376084-7bdd-4093-aab1-f3eda27dab45 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:57.939331 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e376084-7bdd-4093-aab1-f3eda27dab45 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:58.101630 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af29ef40-591c-45d4-bc90-fde36741adbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:58.103114 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29ef40-591c-45d4-bc90-fde36741adbb admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:58.103512 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29ef40-591c-45d4-bc90-fde36741adbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:58.103862 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29ef40-591c-45d4-bc90-fde36741adbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:58.104822 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af29ef40-591c-45d4-bc90-fde36741adbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:58.107151 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af29ef40-591c-45d4-bc90-fde36741adbb admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:58.140180 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1121/2269] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:57 2022] GET /identity/v3/roles => generated 1420 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:58.140932 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da6790-4e9c-4d21-a0a6-bcc8f36cb5be admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:58.160646 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da6790-4e9c-4d21-a0a6-bcc8f36cb5be admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:58.175011 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6146800-4210-4c88-a715-5c6fc623ddf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:58.178984 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6146800-4210-4c88-a715-5c6fc623ddf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:58.179707 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6146800-4210-4c88-a715-5c6fc623ddf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:58.180431 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6146800-4210-4c88-a715-5c6fc623ddf7 admin admin] PATH_INFO: `/v3/projects/8c7bdfa188d54f53b6fa7780406ab470/users/01edd6421bcf4f99ba676bae307fb597/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:58.189791 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6146800-4210-4c88-a715-5c6fc623ddf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c7bdfa188d54f53b6fa7780406ab470, user_id=01edd6421bcf4f99ba676bae307fb597, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:58.194883 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6146800-4210-4c88-a715-5c6fc623ddf7 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:58.233139 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1149/2270] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:58 2022] PUT /identity/v3/projects/8c7bdfa188d54f53b6fa7780406ab470/users/01edd6421bcf4f99ba676bae307fb597/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:58.241359 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea69436-44c0-4eb7-a5cb-ffa28d84191d None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:58.241840 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea69436-44c0-4eb7-a5cb-ffa28d84191d None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:58.242182 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea69436-44c0-4eb7-a5cb-ffa28d84191d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:58.306369 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-6ea69436-44c0-4eb7-a5cb-ffa28d84191d None None] MFA Rules not processed for user `01edd6421bcf4f99ba676bae307fb597`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:58.307853 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-6ea69436-44c0-4eb7-a5cb-ffa28d84191d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:58.398027 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:58.585872 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1122/2271] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:42:58 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:58.595353 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6146800-4210-4c88-a715-5c6fc623ddf7 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:58.600806 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b6146800-4210-4c88-a715-5c6fc623ddf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:58.830894 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6146800-4210-4c88-a715-5c6fc623ddf7 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:58.849301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd3b32-3d29-423a-bae1-7cc018750158 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:58.853135 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd3b32-3d29-423a-bae1-7cc018750158 service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:58.853480 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd3b32-3d29-423a-bae1-7cc018750158 service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:58.853848 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd3b32-3d29-423a-bae1-7cc018750158 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:58.901994 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfd3b32-3d29-423a-bae1-7cc018750158 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:58.903468 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfd3b32-3d29-423a-bae1-7cc018750158 service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:58.926802 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1150/2272] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:58 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:42:59.019585 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea69436-44c0-4eb7-a5cb-ffa28d84191d None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:59.021311 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-6ea69436-44c0-4eb7-a5cb-ffa28d84191d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:59.044995 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea69436-44c0-4eb7-a5cb-ffa28d84191d None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:59.119391 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd98ca2f-52b7-4e3f-85e8-eb8091ca0503 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:59.120077 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98ca2f-52b7-4e3f-85e8-eb8091ca0503 service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:59.120432 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98ca2f-52b7-4e3f-85e8-eb8091ca0503 service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:59.120816 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98ca2f-52b7-4e3f-85e8-eb8091ca0503 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:59.288431 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd98ca2f-52b7-4e3f-85e8-eb8091ca0503 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:59.290098 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd98ca2f-52b7-4e3f-85e8-eb8091ca0503 service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:59.324157 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1123/2273] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:59 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:43:03.536622 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd3b32-3d29-423a-bae1-7cc018750158 service cinder] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:03.663620 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd3b32-3d29-423a-bae1-7cc018750158 service cinder] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:03.669856 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb56736-ca54-4e59-af8b-9b0812afd2fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:03.671320 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb56736-ca54-4e59-af8b-9b0812afd2fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:03.671867 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb56736-ca54-4e59-af8b-9b0812afd2fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:03.672301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb56736-ca54-4e59-af8b-9b0812afd2fb admin admin] PATH_INFO: `/v3/users/d62dbf20083d44d28ea49f77a0a00329` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:03.675045 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb56736-ca54-4e59-af8b-9b0812afd2fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=d62dbf20083d44d28ea49f77a0a00329)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:03.677524 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb56736-ca54-4e59-af8b-9b0812afd2fb admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:03.741842 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:03.759192 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-efb56736-ca54-4e59-af8b-9b0812afd2fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd62dbf20083d44d28ea49f77a0a00329'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:03.780674 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-efb56736-ca54-4e59-af8b-9b0812afd2fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd62dbf20083d44d28ea49f77a0a00329'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:03.791015 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-efb56736-ca54-4e59-af8b-9b0812afd2fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd62dbf20083d44d28ea49f77a0a00329'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:03.791832 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-efb56736-ca54-4e59-af8b-9b0812afd2fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd62dbf20083d44d28ea49f77a0a00329'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:03.792717 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-efb56736-ca54-4e59-af8b-9b0812afd2fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd62dbf20083d44d28ea49f77a0a00329'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:03.827059 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1151/2274] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:43:03 2022] DELETE /identity/v3/users/d62dbf20083d44d28ea49f77a0a00329 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:03.828985 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac44a4a-2cf5-4ca8-aad0-852c65b630fb None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:03.829613 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac44a4a-2cf5-4ca8-aad0-852c65b630fb None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:03.829613 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac44a4a-2cf5-4ca8-aad0-852c65b630fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:03.895023 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-8ac44a4a-2cf5-4ca8-aad0-852c65b630fb None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:03.896781 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-8ac44a4a-2cf5-4ca8-aad0-852c65b630fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:04.096817 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1124/2275] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:43:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:04.107458 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb56736-ca54-4e59-af8b-9b0812afd2fb admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:04.109207 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-efb56736-ca54-4e59-af8b-9b0812afd2fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:04.300797 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb56736-ca54-4e59-af8b-9b0812afd2fb admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:04.308058 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b97c62-4cbc-4fe3-87c0-83ba9fb86d13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:04.308736 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b97c62-4cbc-4fe3-87c0-83ba9fb86d13 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:04.309026 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b97c62-4cbc-4fe3-87c0-83ba9fb86d13 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:04.309358 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b97c62-4cbc-4fe3-87c0-83ba9fb86d13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:04.333392 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b97c62-4cbc-4fe3-87c0-83ba9fb86d13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:04.335109 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b97c62-4cbc-4fe3-87c0-83ba9fb86d13 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:04.359806 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1152/2276] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:43:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:43:04.954953 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac44a4a-2cf5-4ca8-aad0-852c65b630fb None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:04.956790 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-8ac44a4a-2cf5-4ca8-aad0-852c65b630fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:04.984252 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac44a4a-2cf5-4ca8-aad0-852c65b630fb None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:04.990161 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c949fa84-22fe-4abb-9d4b-b602aa0ccfc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:04.991051 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c949fa84-22fe-4abb-9d4b-b602aa0ccfc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:04.991339 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c949fa84-22fe-4abb-9d4b-b602aa0ccfc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:04.991716 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c949fa84-22fe-4abb-9d4b-b602aa0ccfc2 admin admin] PATH_INFO: `/v3/projects/b1b493f59cea4eb48989bbaf7904fa7a` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:04.993081 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c949fa84-22fe-4abb-9d4b-b602aa0ccfc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1b493f59cea4eb48989bbaf7904fa7a)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:04.996696 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c949fa84-22fe-4abb-9d4b-b602aa0ccfc2 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:05.096018 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:05.115256 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c949fa84-22fe-4abb-9d4b-b602aa0ccfc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1b493f59cea4eb48989bbaf7904fa7a', 'request_id': 'req-c949fa84-22fe-4abb-9d4b-b602aa0ccfc2'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:05.133717 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c949fa84-22fe-4abb-9d4b-b602aa0ccfc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1b493f59cea4eb48989bbaf7904fa7a', 'request_id': 'req-c949fa84-22fe-4abb-9d4b-b602aa0ccfc2'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:05.148869 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c949fa84-22fe-4abb-9d4b-b602aa0ccfc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1b493f59cea4eb48989bbaf7904fa7a', 'request_id': 'req-c949fa84-22fe-4abb-9d4b-b602aa0ccfc2'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:05.180786 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c949fa84-22fe-4abb-9d4b-b602aa0ccfc2 admin admin] The token cache is being invalidate because project b1b493f59cea4eb48989bbaf7904fa7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:43:05.181389 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c949fa84-22fe-4abb-9d4b-b602aa0ccfc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:05.182488 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c949fa84-22fe-4abb-9d4b-b602aa0ccfc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:05.186152 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1125/2277] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:43:04 2022] DELETE /identity/v3/projects/b1b493f59cea4eb48989bbaf7904fa7a => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:05.195060 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b97c62-4cbc-4fe3-87c0-83ba9fb86d13 service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:05.211161 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b97c62-4cbc-4fe3-87c0-83ba9fb86d13 service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:05.384581 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaac7c6-b955-4299-b5d9-e65c058a5542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:05.386682 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaac7c6-b955-4299-b5d9-e65c058a5542 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:05.389311 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaac7c6-b955-4299-b5d9-e65c058a5542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:05.389865 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaac7c6-b955-4299-b5d9-e65c058a5542 admin admin] PATH_INFO: `/v3/users/56d6d56bf14047e393c08e46217c3fd9` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:05.394865 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaac7c6-b955-4299-b5d9-e65c058a5542 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56d6d56bf14047e393c08e46217c3fd9)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:05.425160 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaac7c6-b955-4299-b5d9-e65c058a5542 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:05.526747 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:05.559699 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-6eaac7c6-b955-4299-b5d9-e65c058a5542 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56d6d56bf14047e393c08e46217c3fd9'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:05.588486 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-6eaac7c6-b955-4299-b5d9-e65c058a5542 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56d6d56bf14047e393c08e46217c3fd9'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:05.625182 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-6eaac7c6-b955-4299-b5d9-e65c058a5542 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56d6d56bf14047e393c08e46217c3fd9'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:05.627808 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-6eaac7c6-b955-4299-b5d9-e65c058a5542 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56d6d56bf14047e393c08e46217c3fd9'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:05.629159 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-6eaac7c6-b955-4299-b5d9-e65c058a5542 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56d6d56bf14047e393c08e46217c3fd9'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:05.693557 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1153/2278] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:43:05 2022] DELETE /identity/v3/users/56d6d56bf14047e393c08e46217c3fd9 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:06.241841 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7283e7fe-ac08-4756-93af-28c3f3b22a6e None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:06.242885 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7283e7fe-ac08-4756-93af-28c3f3b22a6e None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:06.243362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7283e7fe-ac08-4756-93af-28c3f3b22a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:06.315751 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-7283e7fe-ac08-4756-93af-28c3f3b22a6e None None] MFA Rules not processed for user `ce24dd074ae44e4ea79ccf6451634626`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:06.319131 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-7283e7fe-ac08-4756-93af-28c3f3b22a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:06.491577 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1126/2279] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:43:06 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:06.498811 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaac7c6-b955-4299-b5d9-e65c058a5542 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:06.501702 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-6eaac7c6-b955-4299-b5d9-e65c058a5542 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:06.679606 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaac7c6-b955-4299-b5d9-e65c058a5542 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:06.686437 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535d32e1-ddf4-4d91-a3aa-11d71c119675 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74bb1bc6821b4500b26f160ccae5b7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:06.687120 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535d32e1-ddf4-4d91-a3aa-11d71c119675 service nova] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:06.692648 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535d32e1-ddf4-4d91-a3aa-11d71c119675 service nova] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:06.692945 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535d32e1-ddf4-4d91-a3aa-11d71c119675 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:06.696322 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-535d32e1-ddf4-4d91-a3aa-11d71c119675 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:06.804090 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535d32e1-ddf4-4d91-a3aa-11d71c119675 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:06.806110 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535d32e1-ddf4-4d91-a3aa-11d71c119675 service nova] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:06.839737 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1154/2280] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:43:06 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:43:06.887774 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7283e7fe-ac08-4756-93af-28c3f3b22a6e None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:06.889600 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-7283e7fe-ac08-4756-93af-28c3f3b22a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:07.006135 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7283e7fe-ac08-4756-93af-28c3f3b22a6e None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:07.020940 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feedbf31-0782-41b4-9367-90234d9cddc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:07.022732 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feedbf31-0782-41b4-9367-90234d9cddc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:07.023408 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feedbf31-0782-41b4-9367-90234d9cddc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:07.024484 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feedbf31-0782-41b4-9367-90234d9cddc8 admin admin] PATH_INFO: `/v3/projects/d8f1f250cffa4367ae9a86394d521dcb` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:07.026453 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feedbf31-0782-41b4-9367-90234d9cddc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8f1f250cffa4367ae9a86394d521dcb)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:07.030657 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feedbf31-0782-41b4-9367-90234d9cddc8 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:07.141165 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-feedbf31-0782-41b4-9367-90234d9cddc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8f1f250cffa4367ae9a86394d521dcb', 'request_id': 'req-feedbf31-0782-41b4-9367-90234d9cddc8'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:07.161506 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-feedbf31-0782-41b4-9367-90234d9cddc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8f1f250cffa4367ae9a86394d521dcb', 'request_id': 'req-feedbf31-0782-41b4-9367-90234d9cddc8'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:07.166612 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:07.179055 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-feedbf31-0782-41b4-9367-90234d9cddc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8f1f250cffa4367ae9a86394d521dcb', 'request_id': 'req-feedbf31-0782-41b4-9367-90234d9cddc8'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:07.206174 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-feedbf31-0782-41b4-9367-90234d9cddc8 admin admin] The token cache is being invalidate because project d8f1f250cffa4367ae9a86394d521dcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:43:07.206481 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-feedbf31-0782-41b4-9367-90234d9cddc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:07.207270 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-feedbf31-0782-41b4-9367-90234d9cddc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:07.215900 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535d32e1-ddf4-4d91-a3aa-11d71c119675 service nova] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:07.218096 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1127/2281] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:43:06 2022] DELETE /identity/v3/projects/d8f1f250cffa4367ae9a86394d521dcb => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:07.230783 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535d32e1-ddf4-4d91-a3aa-11d71c119675 service nova] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:07.348212 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:07.389159 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a28c1a8-cfd7-4a99-8f31-d5492f398985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:07.392390 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a28c1a8-cfd7-4a99-8f31-d5492f398985 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:07.392390 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a28c1a8-cfd7-4a99-8f31-d5492f398985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:07.392390 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a28c1a8-cfd7-4a99-8f31-d5492f398985 admin admin] PATH_INFO: `/v3/users/01edd6421bcf4f99ba676bae307fb597` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:07.395116 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a28c1a8-cfd7-4a99-8f31-d5492f398985 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01edd6421bcf4f99ba676bae307fb597)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:07.399003 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a28c1a8-cfd7-4a99-8f31-d5492f398985 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:07.498579 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-1a28c1a8-cfd7-4a99-8f31-d5492f398985 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01edd6421bcf4f99ba676bae307fb597'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:07.526904 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-1a28c1a8-cfd7-4a99-8f31-d5492f398985 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01edd6421bcf4f99ba676bae307fb597'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:07.537670 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-1a28c1a8-cfd7-4a99-8f31-d5492f398985 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01edd6421bcf4f99ba676bae307fb597'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:07.538569 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-1a28c1a8-cfd7-4a99-8f31-d5492f398985 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01edd6421bcf4f99ba676bae307fb597'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:07.539222 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-1a28c1a8-cfd7-4a99-8f31-d5492f398985 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01edd6421bcf4f99ba676bae307fb597'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:07.566810 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1155/2282] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:43:07 2022] DELETE /identity/v3/users/01edd6421bcf4f99ba676bae307fb597 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:08.287674 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feedbf31-0782-41b4-9367-90234d9cddc8 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:08.289165 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-feedbf31-0782-41b4-9367-90234d9cddc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:08.308284 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feedbf31-0782-41b4-9367-90234d9cddc8 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:08.405738 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a875c-9adc-4f0e-89cb-cb585f9c2a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:08.406581 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a875c-9adc-4f0e-89cb-cb585f9c2a6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:08.406897 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a875c-9adc-4f0e-89cb-cb585f9c2a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:08.407219 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a875c-9adc-4f0e-89cb-cb585f9c2a6d admin admin] PATH_INFO: `/v3/users/7fd0bf40927d4d86866e0763842a23b3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:08.408985 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6a875c-9adc-4f0e-89cb-cb585f9c2a6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fd0bf40927d4d86866e0763842a23b3)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:08.411087 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6a875c-9adc-4f0e-89cb-cb585f9c2a6d admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:08.508633 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-dd6a875c-9adc-4f0e-89cb-cb585f9c2a6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fd0bf40927d4d86866e0763842a23b3'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:08.548103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-dd6a875c-9adc-4f0e-89cb-cb585f9c2a6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fd0bf40927d4d86866e0763842a23b3'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:08.548885 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-dd6a875c-9adc-4f0e-89cb-cb585f9c2a6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fd0bf40927d4d86866e0763842a23b3'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:08.560823 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-dd6a875c-9adc-4f0e-89cb-cb585f9c2a6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fd0bf40927d4d86866e0763842a23b3'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:08.561610 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-dd6a875c-9adc-4f0e-89cb-cb585f9c2a6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fd0bf40927d4d86866e0763842a23b3'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:08.593397 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1128/2283] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:43:08 2022] DELETE /identity/v3/users/7fd0bf40927d4d86866e0763842a23b3 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:08.595820 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb054a-00f9-4f7c-bf9d-4547276a43e8 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:08.596770 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb054a-00f9-4f7c-bf9d-4547276a43e8 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:08.597363 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb054a-00f9-4f7c-bf9d-4547276a43e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:08.647808 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-9bcb054a-00f9-4f7c-bf9d-4547276a43e8 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:08.649085 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-9bcb054a-00f9-4f7c-bf9d-4547276a43e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:08.737752 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a875c-9adc-4f0e-89cb-cb585f9c2a6d admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:08.739135 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-dd6a875c-9adc-4f0e-89cb-cb585f9c2a6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:08.788398 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1156/2284] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:43:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:08.813739 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcb054a-00f9-4f7c-bf9d-4547276a43e8 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:08.815746 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-9bcb054a-00f9-4f7c-bf9d-4547276a43e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:08.955717 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcb054a-00f9-4f7c-bf9d-4547276a43e8 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:08.959888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29384b8e-bd8c-4135-98a4-9b2e93f214c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:08.960603 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29384b8e-bd8c-4135-98a4-9b2e93f214c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:08.960937 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29384b8e-bd8c-4135-98a4-9b2e93f214c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:08.961995 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29384b8e-bd8c-4135-98a4-9b2e93f214c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:08.987948 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29384b8e-bd8c-4135-98a4-9b2e93f214c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:08.994736 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29384b8e-bd8c-4135-98a4-9b2e93f214c7 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:09.010293 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a875c-9adc-4f0e-89cb-cb585f9c2a6d admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:09.025311 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded35371-3d56-4760-b26e-142d19d4c735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:09.036152 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1157/2285] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:43:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:43:09.039508 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded35371-3d56-4760-b26e-142d19d4c735 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:09.040071 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded35371-3d56-4760-b26e-142d19d4c735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:09.043554 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded35371-3d56-4760-b26e-142d19d4c735 admin admin] PATH_INFO: `/v3/projects/8c7bdfa188d54f53b6fa7780406ab470` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:09.046812 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded35371-3d56-4760-b26e-142d19d4c735 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c7bdfa188d54f53b6fa7780406ab470)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:09.054175 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded35371-3d56-4760-b26e-142d19d4c735 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:09.078575 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:09.182435 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-ded35371-3d56-4760-b26e-142d19d4c735 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c7bdfa188d54f53b6fa7780406ab470', 'request_id': 'req-ded35371-3d56-4760-b26e-142d19d4c735'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:09.197649 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-ded35371-3d56-4760-b26e-142d19d4c735 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c7bdfa188d54f53b6fa7780406ab470', 'request_id': 'req-ded35371-3d56-4760-b26e-142d19d4c735'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:09.213158 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-ded35371-3d56-4760-b26e-142d19d4c735 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c7bdfa188d54f53b6fa7780406ab470', 'request_id': 'req-ded35371-3d56-4760-b26e-142d19d4c735'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:09.249893 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-ded35371-3d56-4760-b26e-142d19d4c735 admin admin] The token cache is being invalidate because project 8c7bdfa188d54f53b6fa7780406ab470 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:43:09.250486 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-ded35371-3d56-4760-b26e-142d19d4c735 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:09.254913 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-ded35371-3d56-4760-b26e-142d19d4c735 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:09.262686 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1129/2286] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:43:08 2022] DELETE /identity/v3/projects/8c7bdfa188d54f53b6fa7780406ab470 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:43:09.263898 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707a7ec9-1735-46b3-a4cd-994addaa2b52 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:09.264261 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707a7ec9-1735-46b3-a4cd-994addaa2b52 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:09.264592 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707a7ec9-1735-46b3-a4cd-994addaa2b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:09.315859 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:09.389927 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-707a7ec9-1735-46b3-a4cd-994addaa2b52 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:09.398494 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-707a7ec9-1735-46b3-a4cd-994addaa2b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:09.619600 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1158/2287] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:43:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:09.623928 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638aed00-bfa0-4d6f-9f7c-a675f2d575d8 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:09.624468 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638aed00-bfa0-4d6f-9f7c-a675f2d575d8 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:09.624468 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638aed00-bfa0-4d6f-9f7c-a675f2d575d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:09.682025 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-638aed00-bfa0-4d6f-9f7c-a675f2d575d8 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:09.694507 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-638aed00-bfa0-4d6f-9f7c-a675f2d575d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:09.727302 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1130/2288] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:43:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:09.732174 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707a7ec9-1735-46b3-a4cd-994addaa2b52 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:09.758163 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707a7ec9-1735-46b3-a4cd-994addaa2b52 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:09.774482 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e4070a-b644-41fa-be7c-9710d91ae321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:09.775988 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4070a-b644-41fa-be7c-9710d91ae321 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:09.779763 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4070a-b644-41fa-be7c-9710d91ae321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:09.779763 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4070a-b644-41fa-be7c-9710d91ae321 admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:09.784119 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e4070a-b644-41fa-be7c-9710d91ae321 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:43:09.784925 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e4070a-b644-41fa-be7c-9710d91ae321 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:09.786686 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e4070a-b644-41fa-be7c-9710d91ae321 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:09.803911 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1159/2289] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:43:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:43:09.808169 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638aed00-bfa0-4d6f-9f7c-a675f2d575d8 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:09.827154 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638aed00-bfa0-4d6f-9f7c-a675f2d575d8 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:09.842919 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e604730-b597-481a-8dee-2080b3aeea20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:09.843646 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e604730-b597-481a-8dee-2080b3aeea20 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:09.843934 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e604730-b597-481a-8dee-2080b3aeea20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:09.844208 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e604730-b597-481a-8dee-2080b3aeea20 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:09.845677 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e604730-b597-481a-8dee-2080b3aeea20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:09.847774 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e604730-b597-481a-8dee-2080b3aeea20 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:09.905305 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1131/2290] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:43:09 2022] POST /identity/v3/projects => generated 373 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:43:09.906645 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e4070a-b644-41fa-be7c-9710d91ae321 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:09.926920 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e4070a-b644-41fa-be7c-9710d91ae321 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:10.251660 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5520fbbb-e83b-4e68-b5f4-07a02b18e448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:10.253068 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5520fbbb-e83b-4e68-b5f4-07a02b18e448 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:10.253381 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5520fbbb-e83b-4e68-b5f4-07a02b18e448 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:10.253666 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5520fbbb-e83b-4e68-b5f4-07a02b18e448 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:10.254544 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5520fbbb-e83b-4e68-b5f4-07a02b18e448 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:10.256550 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5520fbbb-e83b-4e68-b5f4-07a02b18e448 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:10.315984 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1160/2291] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:43:09 2022] POST /identity/v3/users => generated 287 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:43:10.319701 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e604730-b597-481a-8dee-2080b3aeea20 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:10.333798 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e604730-b597-481a-8dee-2080b3aeea20 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:10.361336 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046d1439-a0f7-4637-b2de-a0e1c7d24a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:10.362386 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d1439-a0f7-4637-b2de-a0e1c7d24a11 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:10.362638 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d1439-a0f7-4637-b2de-a0e1c7d24a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:10.362947 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d1439-a0f7-4637-b2de-a0e1c7d24a11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:10.363714 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046d1439-a0f7-4637-b2de-a0e1c7d24a11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:10.365370 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046d1439-a0f7-4637-b2de-a0e1c7d24a11 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:10.404898 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1132/2292] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:43:10 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:43:10.409474 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5520fbbb-e83b-4e68-b5f4-07a02b18e448 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:10.422667 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5520fbbb-e83b-4e68-b5f4-07a02b18e448 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:10.436805 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56fe112-4de8-4217-8b8d-67743565f5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:10.437604 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56fe112-4de8-4217-8b8d-67743565f5f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:10.438205 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56fe112-4de8-4217-8b8d-67743565f5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:10.438690 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56fe112-4de8-4217-8b8d-67743565f5f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:10.439542 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56fe112-4de8-4217-8b8d-67743565f5f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:10.442116 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56fe112-4de8-4217-8b8d-67743565f5f7 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:10.471731 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1161/2293] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:43:10 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:43:10.472533 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046d1439-a0f7-4637-b2de-a0e1c7d24a11 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:10.486754 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046d1439-a0f7-4637-b2de-a0e1c7d24a11 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:10.507710 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e1e50-8d93-4cdd-9e2a-9f66cefe950e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:10.508743 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e1e50-8d93-4cdd-9e2a-9f66cefe950e admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:10.509136 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e1e50-8d93-4cdd-9e2a-9f66cefe950e admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:10.509651 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e1e50-8d93-4cdd-9e2a-9f66cefe950e admin admin] PATH_INFO: `/v3/projects/b0eee67a72f743cd816e9136c09c1ea0/users/11fb01f7009d4e1993ab86c6a93b4830/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:10.568059 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214e1e50-8d93-4cdd-9e2a-9f66cefe950e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0eee67a72f743cd816e9136c09c1ea0, user_id=11fb01f7009d4e1993ab86c6a93b4830, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:10.575178 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214e1e50-8d93-4cdd-9e2a-9f66cefe950e admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:10.613874 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1133/2294] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:43:10 2022] PUT /identity/v3/projects/b0eee67a72f743cd816e9136c09c1ea0/users/11fb01f7009d4e1993ab86c6a93b4830/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:43:10.614966 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ccd2a-877d-4c4b-bb2e-ae57bd3e2222 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:10.615348 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ccd2a-877d-4c4b-bb2e-ae57bd3e2222 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:10.615724 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ccd2a-877d-4c4b-bb2e-ae57bd3e2222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:10.645940 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-d82ccd2a-877d-4c4b-bb2e-ae57bd3e2222 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:10.647480 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-d82ccd2a-877d-4c4b-bb2e-ae57bd3e2222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:10.756083 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:10.799584 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1162/2295] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:43:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:10.825653 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e1e50-8d93-4cdd-9e2a-9f66cefe950e admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:10.828775 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-214e1e50-8d93-4cdd-9e2a-9f66cefe950e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:10.937390 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82ccd2a-877d-4c4b-bb2e-ae57bd3e2222 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:10.939610 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-d82ccd2a-877d-4c4b-bb2e-ae57bd3e2222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:10.974410 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e1e50-8d93-4cdd-9e2a-9f66cefe950e admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:10.983949 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2365301c-2c09-42e2-b012-254852290b7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:10.984627 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365301c-2c09-42e2-b012-254852290b7e service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:10.985485 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365301c-2c09-42e2-b012-254852290b7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:10.985803 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365301c-2c09-42e2-b012-254852290b7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:11.004390 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82ccd2a-877d-4c4b-bb2e-ae57bd3e2222 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:11.020156 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123745b6-996a-4e75-9413-2602ce5dcf79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:11.021174 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123745b6-996a-4e75-9413-2602ce5dcf79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:11.021556 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123745b6-996a-4e75-9413-2602ce5dcf79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:11.021995 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123745b6-996a-4e75-9413-2602ce5dcf79 admin admin] PATH_INFO: `/v3/projects/19733ce9e7f14e8da367fa7a04dfdc74` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:11.024893 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123745b6-996a-4e75-9413-2602ce5dcf79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19733ce9e7f14e8da367fa7a04dfdc74)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:11.027090 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123745b6-996a-4e75-9413-2602ce5dcf79 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:11.035454 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2365301c-2c09-42e2-b012-254852290b7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:11.037864 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2365301c-2c09-42e2-b012-254852290b7e service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:11.062996 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1134/2296] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:43:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:43:11.120676 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-123745b6-996a-4e75-9413-2602ce5dcf79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19733ce9e7f14e8da367fa7a04dfdc74', 'request_id': 'req-123745b6-996a-4e75-9413-2602ce5dcf79'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:11.136768 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-123745b6-996a-4e75-9413-2602ce5dcf79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19733ce9e7f14e8da367fa7a04dfdc74', 'request_id': 'req-123745b6-996a-4e75-9413-2602ce5dcf79'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:11.144843 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-123745b6-996a-4e75-9413-2602ce5dcf79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19733ce9e7f14e8da367fa7a04dfdc74', 'request_id': 'req-123745b6-996a-4e75-9413-2602ce5dcf79'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:11.169022 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-123745b6-996a-4e75-9413-2602ce5dcf79 admin admin] The token cache is being invalidate because project 19733ce9e7f14e8da367fa7a04dfdc74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:43:11.169312 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-123745b6-996a-4e75-9413-2602ce5dcf79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:11.170555 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-123745b6-996a-4e75-9413-2602ce5dcf79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:11.185721 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1163/2297] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:43:10 2022] DELETE /identity/v3/projects/19733ce9e7f14e8da367fa7a04dfdc74 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:43:12.729921 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d81bb-7a50-4420-8691-dc1ef67b8433 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:12.730863 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d81bb-7a50-4420-8691-dc1ef67b8433 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:12.731249 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d81bb-7a50-4420-8691-dc1ef67b8433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:12.774143 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-906d81bb-7a50-4420-8691-dc1ef67b8433 None None] MFA Rules not processed for user `11fb01f7009d4e1993ab86c6a93b4830`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:12.776035 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-906d81bb-7a50-4420-8691-dc1ef67b8433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:12.825997 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:13.008915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1135/2298] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:43:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:13.013618 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123745b6-996a-4e75-9413-2602ce5dcf79 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:13.015292 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-123745b6-996a-4e75-9413-2602ce5dcf79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:13.090152 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123745b6-996a-4e75-9413-2602ce5dcf79 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:13.095759 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf9001-b4e3-4f89-b324-ecab6ca46c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:13.096438 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf9001-b4e3-4f89-b324-ecab6ca46c61 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:13.096923 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf9001-b4e3-4f89-b324-ecab6ca46c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:13.097216 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf9001-b4e3-4f89-b324-ecab6ca46c61 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:13.097909 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaf9001-b4e3-4f89-b324-ecab6ca46c61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:13.101586 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaf9001-b4e3-4f89-b324-ecab6ca46c61 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:13.140865 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1164/2299] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:43:13 2022] POST /identity/v3/projects => generated 371 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:43:13.145115 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906d81bb-7a50-4420-8691-dc1ef67b8433 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:13.157953 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906d81bb-7a50-4420-8691-dc1ef67b8433 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:13.250292 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11955447-e5a5-43e6-8317-95b919bddf2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:13.252330 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11955447-e5a5-43e6-8317-95b919bddf2f admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:13.252913 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11955447-e5a5-43e6-8317-95b919bddf2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:13.253457 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11955447-e5a5-43e6-8317-95b919bddf2f admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:13.254973 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11955447-e5a5-43e6-8317-95b919bddf2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:13.258561 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11955447-e5a5-43e6-8317-95b919bddf2f admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:13.299324 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1136/2300] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:43:13 2022] POST /identity/v3/users => generated 292 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:43:13.305672 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf9001-b4e3-4f89-b324-ecab6ca46c61 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:13.320601 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf9001-b4e3-4f89-b324-ecab6ca46c61 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:13.330640 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd06cd69-4c2e-4674-81dc-b9804329eb5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:13.331849 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06cd69-4c2e-4674-81dc-b9804329eb5a admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:13.332206 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06cd69-4c2e-4674-81dc-b9804329eb5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:13.332595 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06cd69-4c2e-4674-81dc-b9804329eb5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:13.333520 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd06cd69-4c2e-4674-81dc-b9804329eb5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:13.336287 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd06cd69-4c2e-4674-81dc-b9804329eb5a admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:13.361882 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1165/2301] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:43:13 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:43:13.364521 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11955447-e5a5-43e6-8317-95b919bddf2f admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:13.377767 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11955447-e5a5-43e6-8317-95b919bddf2f admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:13.383830 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f891e225-e42e-4c8e-aa9a-98929f70e1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:13.385200 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f891e225-e42e-4c8e-aa9a-98929f70e1cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:13.385654 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f891e225-e42e-4c8e-aa9a-98929f70e1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:13.386113 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f891e225-e42e-4c8e-aa9a-98929f70e1cc admin admin] PATH_INFO: `/v3/projects/c35a508a79194609bed2df14f0fb07f2/users/01c1552fdb784dd499b02a8c3bceaab8/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:13.408697 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f891e225-e42e-4c8e-aa9a-98929f70e1cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c35a508a79194609bed2df14f0fb07f2, user_id=01c1552fdb784dd499b02a8c3bceaab8, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:13.412511 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f891e225-e42e-4c8e-aa9a-98929f70e1cc admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:13.431957 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1137/2302] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:43:13 2022] PUT /identity/v3/projects/c35a508a79194609bed2df14f0fb07f2/users/01c1552fdb784dd499b02a8c3bceaab8/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:13.433303 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd06cd69-4c2e-4674-81dc-b9804329eb5a admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:13.446054 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd06cd69-4c2e-4674-81dc-b9804329eb5a admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:13.516969 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:13.523006 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a0a261-9fb3-4ca8-8076-7c2f90818e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:13.523698 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0a261-9fb3-4ca8-8076-7c2f90818e3a admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:13.524364 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0a261-9fb3-4ca8-8076-7c2f90818e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:13.524857 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0a261-9fb3-4ca8-8076-7c2f90818e3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:13.526178 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a0a261-9fb3-4ca8-8076-7c2f90818e3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:13.528577 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a0a261-9fb3-4ca8-8076-7c2f90818e3a admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:13.550916 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1166/2303] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:43:13 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:43:13.554876 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f891e225-e42e-4c8e-aa9a-98929f70e1cc admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:13.568021 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f891e225-e42e-4c8e-aa9a-98929f70e1cc admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:13.579371 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09916652-14bb-4fe0-b518-fbf98e391a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:13.579654 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09916652-14bb-4fe0-b518-fbf98e391a55 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:13.580024 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09916652-14bb-4fe0-b518-fbf98e391a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:13.580377 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09916652-14bb-4fe0-b518-fbf98e391a55 admin admin] PATH_INFO: `/v3/projects/c35a508a79194609bed2df14f0fb07f2/users/01c1552fdb784dd499b02a8c3bceaab8/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:13.582809 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09916652-14bb-4fe0-b518-fbf98e391a55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c35a508a79194609bed2df14f0fb07f2, user_id=01c1552fdb784dd499b02a8c3bceaab8, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:13.585864 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09916652-14bb-4fe0-b518-fbf98e391a55 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:13.599911 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1138/2304] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:43:13 2022] PUT /identity/v3/projects/c35a508a79194609bed2df14f0fb07f2/users/01c1552fdb784dd499b02a8c3bceaab8/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:14.476340 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9798cc-393d-4e31-b1f1-d4907c7f3f56 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:14.476915 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9798cc-393d-4e31-b1f1-d4907c7f3f56 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:14.476915 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9798cc-393d-4e31-b1f1-d4907c7f3f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:14.553471 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-dd9798cc-393d-4e31-b1f1-d4907c7f3f56 None None] MFA Rules not processed for user `01c1552fdb784dd499b02a8c3bceaab8`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:14.555333 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-dd9798cc-393d-4e31-b1f1-d4907c7f3f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:14.736504 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1167/2305] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:43:14 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:14.757868 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09916652-14bb-4fe0-b518-fbf98e391a55 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:14.760283 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-09916652-14bb-4fe0-b518-fbf98e391a55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:14.854937 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09916652-14bb-4fe0-b518-fbf98e391a55 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:14.858983 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e23c2-2020-478d-8616-9358ad7fc9c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:14.861196 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e23c2-2020-478d-8616-9358ad7fc9c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:14.861496 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e23c2-2020-478d-8616-9358ad7fc9c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:14.861832 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e23c2-2020-478d-8616-9358ad7fc9c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:14.908190 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:14.954419 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e23c2-2020-478d-8616-9358ad7fc9c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:14.956047 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e23c2-2020-478d-8616-9358ad7fc9c4 service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:14.968658 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1139/2306] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:43:14 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:43:16.461329 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9798cc-393d-4e31-b1f1-d4907c7f3f56 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:16.473890 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9798cc-393d-4e31-b1f1-d4907c7f3f56 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:16.480528 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ef409f-5cfd-45fa-87c6-f857393e5630 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:16.481297 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef409f-5cfd-45fa-87c6-f857393e5630 service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:16.481668 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef409f-5cfd-45fa-87c6-f857393e5630 service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:16.482056 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef409f-5cfd-45fa-87c6-f857393e5630 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:16.495981 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ef409f-5cfd-45fa-87c6-f857393e5630 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:16.498280 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ef409f-5cfd-45fa-87c6-f857393e5630 service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:16.514706 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1168/2307] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:43:16 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:43:24.600516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b324087-9246-438b-bcbe-1f9b7172673c None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:24.601076 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b324087-9246-438b-bcbe-1f9b7172673c None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:24.601267 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b324087-9246-438b-bcbe-1f9b7172673c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:24.636623 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-3b324087-9246-438b-bcbe-1f9b7172673c None None] MFA Rules not processed for user `85d9c4c57681447c898ebfd7990b05b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:24.638697 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-3b324087-9246-438b-bcbe-1f9b7172673c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:24.714116 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1140/2308] 10.222.0.13 () {58 vars in 1009 bytes} [Fri Jun 24 08:43:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:24.726560 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ef409f-5cfd-45fa-87c6-f857393e5630 service cinder] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:24.727960 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-51ef409f-5cfd-45fa-87c6-f857393e5630 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:24.800184 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ef409f-5cfd-45fa-87c6-f857393e5630 service cinder] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:24.805221 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b17fb3-22c3-4d09-9946-924a7df05e7f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec8c2f113cf41a0939b81f38fbee813', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:24.805941 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b17fb3-22c3-4d09-9946-924a7df05e7f service swift] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:24.806228 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b17fb3-22c3-4d09-9946-924a7df05e7f service swift] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:24.806502 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b17fb3-22c3-4d09-9946-924a7df05e7f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:24.815775 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b17fb3-22c3-4d09-9946-924a7df05e7f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:24.817437 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b17fb3-22c3-4d09-9946-924a7df05e7f service swift] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:24.826233 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1169/2309] 10.222.0.13 () {60 vars in 1384 bytes} [Fri Jun 24 08:43:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 08:43:29.012652 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d555d2d3-3a45-4f5c-b877-d7b30ffaf0e8 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:29.013250 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d555d2d3-3a45-4f5c-b877-d7b30ffaf0e8 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:29.013339 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d555d2d3-3a45-4f5c-b877-d7b30ffaf0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:29.032967 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-d555d2d3-3a45-4f5c-b877-d7b30ffaf0e8 None None] MFA Rules not processed for user `85d9c4c57681447c898ebfd7990b05b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:29.034415 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d555d2d3-3a45-4f5c-b877-d7b30ffaf0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:29.042728 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1141/2310] 10.222.0.13 () {58 vars in 967 bytes} [Fri Jun 24 08:43:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:29.063051 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b17fb3-22c3-4d09-9946-924a7df05e7f service swift] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:29.077399 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b17fb3-22c3-4d09-9946-924a7df05e7f service swift] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:29.083668 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db5ee6-b410-4328-bddc-c537357101a7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec8c2f113cf41a0939b81f38fbee813', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:29.084315 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db5ee6-b410-4328-bddc-c537357101a7 service swift] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:29.084596 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db5ee6-b410-4328-bddc-c537357101a7 service swift] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:29.085052 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db5ee6-b410-4328-bddc-c537357101a7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:29.098411 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5db5ee6-b410-4328-bddc-c537357101a7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:29.100297 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5db5ee6-b410-4328-bddc-c537357101a7 service swift] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:29.112014 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1170/2311] 10.222.0.13 () {60 vars in 1384 bytes} [Fri Jun 24 08:43:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 08:43:31.170333 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d555d2d3-3a45-4f5c-b877-d7b30ffaf0e8 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:31.284841 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d555d2d3-3a45-4f5c-b877-d7b30ffaf0e8 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:31.292329 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d05338-1d13-435b-a518-f3653bd46ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:31.293378 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d05338-1d13-435b-a518-f3653bd46ff7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:31.293667 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d05338-1d13-435b-a518-f3653bd46ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:31.293968 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d05338-1d13-435b-a518-f3653bd46ff7 admin admin] PATH_INFO: `/v3/users/11fb01f7009d4e1993ab86c6a93b4830` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:31.295192 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d05338-1d13-435b-a518-f3653bd46ff7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11fb01f7009d4e1993ab86c6a93b4830)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:31.297200 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d05338-1d13-435b-a518-f3653bd46ff7 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:31.389717 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-b5d05338-1d13-435b-a518-f3653bd46ff7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11fb01f7009d4e1993ab86c6a93b4830'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:31.403317 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:31.416147 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-b5d05338-1d13-435b-a518-f3653bd46ff7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11fb01f7009d4e1993ab86c6a93b4830'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:31.418850 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-b5d05338-1d13-435b-a518-f3653bd46ff7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11fb01f7009d4e1993ab86c6a93b4830'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:31.431377 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-b5d05338-1d13-435b-a518-f3653bd46ff7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11fb01f7009d4e1993ab86c6a93b4830'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:31.432351 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-b5d05338-1d13-435b-a518-f3653bd46ff7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11fb01f7009d4e1993ab86c6a93b4830'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:31.451455 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1142/2312] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:43:31 2022] DELETE /identity/v3/users/11fb01f7009d4e1993ab86c6a93b4830 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:31.675338 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db5ee6-b410-4328-bddc-c537357101a7 service swift] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:31.676823 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-c5db5ee6-b410-4328-bddc-c537357101a7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:31.697300 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db5ee6-b410-4328-bddc-c537357101a7 service swift] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:31.755057 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:31.776112 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffffbd1-b80e-4edd-a0cc-b4b82b635331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:31.777437 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffffbd1-b80e-4edd-a0cc-b4b82b635331 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:31.778083 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffffbd1-b80e-4edd-a0cc-b4b82b635331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:31.778559 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffffbd1-b80e-4edd-a0cc-b4b82b635331 admin admin] PATH_INFO: `/v3/users/740d843e5abb439b954996aaf841ed63` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:31.781933 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffffbd1-b80e-4edd-a0cc-b4b82b635331 admin admin] RBAC: Authorizing `identity:delete_user(user_id=740d843e5abb439b954996aaf841ed63)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:31.782084 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d05338-1d13-435b-a518-f3653bd46ff7 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:31.783607 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-b5d05338-1d13-435b-a518-f3653bd46ff7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:31.784625 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffffbd1-b80e-4edd-a0cc-b4b82b635331 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:31.823624 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d05338-1d13-435b-a518-f3653bd46ff7 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:31.831433 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753c1140-b1b9-4315-86b4-460282359802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:31.832674 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c1140-b1b9-4315-86b4-460282359802 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:31.833090 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c1140-b1b9-4315-86b4-460282359802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:31.833434 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c1140-b1b9-4315-86b4-460282359802 admin admin] PATH_INFO: `/v3/projects/b0eee67a72f743cd816e9136c09c1ea0` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:31.834823 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753c1140-b1b9-4315-86b4-460282359802 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0eee67a72f743cd816e9136c09c1ea0)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:31.837029 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753c1140-b1b9-4315-86b4-460282359802 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:31.857339 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0ffffbd1-b80e-4edd-a0cc-b4b82b635331 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '740d843e5abb439b954996aaf841ed63'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:31.875802 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0ffffbd1-b80e-4edd-a0cc-b4b82b635331 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '740d843e5abb439b954996aaf841ed63'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:31.885916 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0ffffbd1-b80e-4edd-a0cc-b4b82b635331 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '740d843e5abb439b954996aaf841ed63'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:31.887124 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0ffffbd1-b80e-4edd-a0cc-b4b82b635331 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '740d843e5abb439b954996aaf841ed63'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:31.889101 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0ffffbd1-b80e-4edd-a0cc-b4b82b635331 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '740d843e5abb439b954996aaf841ed63'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:31.904917 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-753c1140-b1b9-4315-86b4-460282359802 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0eee67a72f743cd816e9136c09c1ea0', 'request_id': 'req-753c1140-b1b9-4315-86b4-460282359802'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:31.910348 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1171/2313] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:43:31 2022] DELETE /identity/v3/users/740d843e5abb439b954996aaf841ed63 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:31.933519 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-753c1140-b1b9-4315-86b4-460282359802 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0eee67a72f743cd816e9136c09c1ea0', 'request_id': 'req-753c1140-b1b9-4315-86b4-460282359802'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:31.947548 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-753c1140-b1b9-4315-86b4-460282359802 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0eee67a72f743cd816e9136c09c1ea0', 'request_id': 'req-753c1140-b1b9-4315-86b4-460282359802'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:31.972332 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-753c1140-b1b9-4315-86b4-460282359802 admin admin] The token cache is being invalidate because project b0eee67a72f743cd816e9136c09c1ea0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:43:31.974647 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-753c1140-b1b9-4315-86b4-460282359802 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:31.975657 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-753c1140-b1b9-4315-86b4-460282359802 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:31.980736 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1143/2314] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:43:31 2022] DELETE /identity/v3/projects/b0eee67a72f743cd816e9136c09c1ea0 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:31.982021 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffffbd1-b80e-4edd-a0cc-b4b82b635331 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:31.983713 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-0ffffbd1-b80e-4edd-a0cc-b4b82b635331 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:32.115533 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffffbd1-b80e-4edd-a0cc-b4b82b635331 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:32.130446 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837ff09c-6ec9-48a6-bfd2-e7c79046c17a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:32.131346 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ff09c-6ec9-48a6-bfd2-e7c79046c17a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:32.131877 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ff09c-6ec9-48a6-bfd2-e7c79046c17a admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:32.132160 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ff09c-6ec9-48a6-bfd2-e7c79046c17a admin admin] PATH_INFO: `/v3/users/01c1552fdb784dd499b02a8c3bceaab8` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:32.133479 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837ff09c-6ec9-48a6-bfd2-e7c79046c17a admin admin] RBAC: Authorizing `identity:delete_user(user_id=01c1552fdb784dd499b02a8c3bceaab8)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:32.136099 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837ff09c-6ec9-48a6-bfd2-e7c79046c17a admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:32.211777 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753c1140-b1b9-4315-86b4-460282359802 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:32.212982 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-753c1140-b1b9-4315-86b4-460282359802 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:32.229540 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-837ff09c-6ec9-48a6-bfd2-e7c79046c17a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01c1552fdb784dd499b02a8c3bceaab8'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:32.255928 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:32.259347 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-837ff09c-6ec9-48a6-bfd2-e7c79046c17a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01c1552fdb784dd499b02a8c3bceaab8'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:32.283172 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-837ff09c-6ec9-48a6-bfd2-e7c79046c17a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01c1552fdb784dd499b02a8c3bceaab8'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:32.285187 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-837ff09c-6ec9-48a6-bfd2-e7c79046c17a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01c1552fdb784dd499b02a8c3bceaab8'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:32.286538 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-837ff09c-6ec9-48a6-bfd2-e7c79046c17a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01c1552fdb784dd499b02a8c3bceaab8'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:32.314575 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1172/2315] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:43:31 2022] DELETE /identity/v3/users/01c1552fdb784dd499b02a8c3bceaab8 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:43:32.316101 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753c1140-b1b9-4315-86b4-460282359802 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:32.407344 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdd994a-fcf9-4115-b803-c4a4129c9b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:32.408407 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd994a-fcf9-4115-b803-c4a4129c9b52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:32.408808 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd994a-fcf9-4115-b803-c4a4129c9b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:32.409569 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd994a-fcf9-4115-b803-c4a4129c9b52 admin admin] PATH_INFO: `/v3/projects/91137dbd48fe4e6d986c350c8435bbab` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:32.411099 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdd994a-fcf9-4115-b803-c4a4129c9b52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91137dbd48fe4e6d986c350c8435bbab)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:32.413191 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdd994a-fcf9-4115-b803-c4a4129c9b52 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:32.415977 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:32.494511 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-efdd994a-fcf9-4115-b803-c4a4129c9b52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91137dbd48fe4e6d986c350c8435bbab', 'request_id': 'req-efdd994a-fcf9-4115-b803-c4a4129c9b52'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:32.506788 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-efdd994a-fcf9-4115-b803-c4a4129c9b52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91137dbd48fe4e6d986c350c8435bbab', 'request_id': 'req-efdd994a-fcf9-4115-b803-c4a4129c9b52'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:32.513722 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-efdd994a-fcf9-4115-b803-c4a4129c9b52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91137dbd48fe4e6d986c350c8435bbab', 'request_id': 'req-efdd994a-fcf9-4115-b803-c4a4129c9b52'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:32.531963 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-efdd994a-fcf9-4115-b803-c4a4129c9b52 admin admin] The token cache is being invalidate because project 91137dbd48fe4e6d986c350c8435bbab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:43:32.532399 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-efdd994a-fcf9-4115-b803-c4a4129c9b52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:32.533109 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-efdd994a-fcf9-4115-b803-c4a4129c9b52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:32.534861 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1144/2316] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:43:32 2022] DELETE /identity/v3/projects/91137dbd48fe4e6d986c350c8435bbab => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:32.539932 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef478f-75e0-4377-aad2-55e3b9c0977f None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:32.540224 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef478f-75e0-4377-aad2-55e3b9c0977f None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:32.540660 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef478f-75e0-4377-aad2-55e3b9c0977f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:32.570718 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-58ef478f-75e0-4377-aad2-55e3b9c0977f None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:32.571954 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-58ef478f-75e0-4377-aad2-55e3b9c0977f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:32.603606 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdd994a-fcf9-4115-b803-c4a4129c9b52 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:32.605726 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-efdd994a-fcf9-4115-b803-c4a4129c9b52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:32.688524 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1173/2317] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:43:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:32.695724 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e657e11-b928-47f3-b64b-0528eee68729 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:32.696131 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e657e11-b928-47f3-b64b-0528eee68729 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:32.696430 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e657e11-b928-47f3-b64b-0528eee68729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:32.712167 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdd994a-fcf9-4115-b803-c4a4129c9b52 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:32.717292 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b1effc-ce8a-4ea0-b262-d787dbab521f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:32.718192 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b1effc-ce8a-4ea0-b262-d787dbab521f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:32.718479 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b1effc-ce8a-4ea0-b262-d787dbab521f admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:32.718796 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b1effc-ce8a-4ea0-b262-d787dbab521f admin admin] PATH_INFO: `/v3/projects/c35a508a79194609bed2df14f0fb07f2` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:32.719028 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-6e657e11-b928-47f3-b64b-0528eee68729 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:32.720004 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b1effc-ce8a-4ea0-b262-d787dbab521f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c35a508a79194609bed2df14f0fb07f2)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:32.720461 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-6e657e11-b928-47f3-b64b-0528eee68729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:32.721995 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b1effc-ce8a-4ea0-b262-d787dbab521f admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:32.730498 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1174/2318] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:43:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:43:32.735095 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e657e11-b928-47f3-b64b-0528eee68729 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:32.759093 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e657e11-b928-47f3-b64b-0528eee68729 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:32.767922 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c650030-7d23-4b7b-bfef-b398157e8b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:32.768614 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c650030-7d23-4b7b-bfef-b398157e8b05 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:32.769817 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c650030-7d23-4b7b-bfef-b398157e8b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:32.770112 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c650030-7d23-4b7b-bfef-b398157e8b05 admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:32.771142 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c650030-7d23-4b7b-bfef-b398157e8b05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:43:32.771453 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c650030-7d23-4b7b-bfef-b398157e8b05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:32.779764 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c650030-7d23-4b7b-bfef-b398157e8b05 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:32.810353 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1175/2319] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:43:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:43:32.815179 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c6b1effc-ce8a-4ea0-b262-d787dbab521f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c35a508a79194609bed2df14f0fb07f2', 'request_id': 'req-c6b1effc-ce8a-4ea0-b262-d787dbab521f'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:32.817730 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c650030-7d23-4b7b-bfef-b398157e8b05 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:32.829821 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c6b1effc-ce8a-4ea0-b262-d787dbab521f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c35a508a79194609bed2df14f0fb07f2', 'request_id': 'req-c6b1effc-ce8a-4ea0-b262-d787dbab521f'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:32.835353 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c650030-7d23-4b7b-bfef-b398157e8b05 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:32.842977 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c6b1effc-ce8a-4ea0-b262-d787dbab521f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c35a508a79194609bed2df14f0fb07f2', 'request_id': 'req-c6b1effc-ce8a-4ea0-b262-d787dbab521f'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:32.867475 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c6b1effc-ce8a-4ea0-b262-d787dbab521f admin admin] The token cache is being invalidate because project c35a508a79194609bed2df14f0fb07f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:43:32.867774 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c6b1effc-ce8a-4ea0-b262-d787dbab521f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:32.868510 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c6b1effc-ce8a-4ea0-b262-d787dbab521f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:32.870299 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1145/2320] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:43:32 2022] DELETE /identity/v3/projects/c35a508a79194609bed2df14f0fb07f2 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:43:32.876363 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2047d8-9f93-47be-8f3c-48eb092f00f1 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:32.876649 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2047d8-9f93-47be-8f3c-48eb092f00f1 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:32.876929 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2047d8-9f93-47be-8f3c-48eb092f00f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:32.915239 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-5d2047d8-9f93-47be-8f3c-48eb092f00f1 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:32.916868 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-5d2047d8-9f93-47be-8f3c-48eb092f00f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:32.945445 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b8a5bd-2c47-4c0e-a908-ead651b0ff50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:32.947560 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8a5bd-2c47-4c0e-a908-ead651b0ff50 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:32.948127 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8a5bd-2c47-4c0e-a908-ead651b0ff50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:32.948721 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8a5bd-2c47-4c0e-a908-ead651b0ff50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:32.949573 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b8a5bd-2c47-4c0e-a908-ead651b0ff50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:32.951686 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b8a5bd-2c47-4c0e-a908-ead651b0ff50 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:32.981608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:32.993779 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1176/2321] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:43:32 2022] POST /identity/v3/projects => generated 381 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:43:33.007387 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b8a5bd-2c47-4c0e-a908-ead651b0ff50 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:33.009958 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-d5b8a5bd-2c47-4c0e-a908-ead651b0ff50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:33.090626 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1146/2322] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:43:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:43:33.092346 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:33.098523 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623ab3d-14a3-4458-b15c-2919906cb435 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:33.098817 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623ab3d-14a3-4458-b15c-2919906cb435 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:33.099112 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623ab3d-14a3-4458-b15c-2919906cb435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:33.118741 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b8a5bd-2c47-4c0e-a908-ead651b0ff50 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:33.120330 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-0623ab3d-14a3-4458-b15c-2919906cb435 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:33.121545 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-0623ab3d-14a3-4458-b15c-2919906cb435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:33.124217 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a12efe-6f29-4904-80e9-d04c3bad23ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:33.125471 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a12efe-6f29-4904-80e9-d04c3bad23ab admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:33.125871 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a12efe-6f29-4904-80e9-d04c3bad23ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:33.126162 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a12efe-6f29-4904-80e9-d04c3bad23ab admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:33.127364 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a12efe-6f29-4904-80e9-d04c3bad23ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:33.128816 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a12efe-6f29-4904-80e9-d04c3bad23ab admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:33.134345 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1147/2323] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:43:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:43:33.137654 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0623ab3d-14a3-4458-b15c-2919906cb435 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:33.149103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0623ab3d-14a3-4458-b15c-2919906cb435 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:33.158419 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1177/2324] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:43:33 2022] POST /identity/v3/users => generated 291 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:43:33.163853 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b737dd-71a3-4146-ae7e-a54406a73137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:33.164706 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b737dd-71a3-4146-ae7e-a54406a73137 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:33.165061 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b737dd-71a3-4146-ae7e-a54406a73137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:33.165434 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b737dd-71a3-4146-ae7e-a54406a73137 admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:33.166447 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b737dd-71a3-4146-ae7e-a54406a73137 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:43:33.166749 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b737dd-71a3-4146-ae7e-a54406a73137 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:33.167267 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a12efe-6f29-4904-80e9-d04c3bad23ab admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:33.171334 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b737dd-71a3-4146-ae7e-a54406a73137 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:33.195417 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a12efe-6f29-4904-80e9-d04c3bad23ab admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:33.202397 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f37cfb2-f1eb-4e3e-9129-931de75540e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:33.203149 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f37cfb2-f1eb-4e3e-9129-931de75540e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:33.203458 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f37cfb2-f1eb-4e3e-9129-931de75540e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:33.203717 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f37cfb2-f1eb-4e3e-9129-931de75540e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:33.207485 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f37cfb2-f1eb-4e3e-9129-931de75540e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:33.207658 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1148/2325] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:43:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:43:33.208397 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f37cfb2-f1eb-4e3e-9129-931de75540e0 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:33.209125 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b737dd-71a3-4146-ae7e-a54406a73137 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:33.224013 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b737dd-71a3-4146-ae7e-a54406a73137 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:33.229829 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b61dad-160a-4ad2-a416-fbcdf6b310c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:33.230649 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b61dad-160a-4ad2-a416-fbcdf6b310c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:33.231110 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b61dad-160a-4ad2-a416-fbcdf6b310c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:33.236506 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1178/2326] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:43:33 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:43:33.241193 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b61dad-160a-4ad2-a416-fbcdf6b310c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:33.242056 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f37cfb2-f1eb-4e3e-9129-931de75540e0 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:33.242211 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b61dad-160a-4ad2-a416-fbcdf6b310c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:33.244679 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b61dad-160a-4ad2-a416-fbcdf6b310c7 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:33.257989 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f37cfb2-f1eb-4e3e-9129-931de75540e0 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:33.268217 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b5c1f7-415d-4989-a0aa-d7357aa51966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:33.269057 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5c1f7-415d-4989-a0aa-d7357aa51966 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:33.269349 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5c1f7-415d-4989-a0aa-d7357aa51966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:33.269626 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5c1f7-415d-4989-a0aa-d7357aa51966 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:33.270402 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b5c1f7-415d-4989-a0aa-d7357aa51966 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:33.272523 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b5c1f7-415d-4989-a0aa-d7357aa51966 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:33.289651 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1179/2327] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:43:33 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:43:33.290487 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b5c1f7-415d-4989-a0aa-d7357aa51966 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:33.295838 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1149/2328] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:43:33 2022] POST /identity/v3/projects => generated 377 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:43:33.297835 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b61dad-160a-4ad2-a416-fbcdf6b310c7 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:33.303079 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b5c1f7-415d-4989-a0aa-d7357aa51966 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:33.387470 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b61dad-160a-4ad2-a416-fbcdf6b310c7 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:33.392434 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb0c36-6de7-467b-a36c-4075ac8dac4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:33.393420 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb0c36-6de7-467b-a36c-4075ac8dac4c admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:33.393833 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb0c36-6de7-467b-a36c-4075ac8dac4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:33.394214 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb0c36-6de7-467b-a36c-4075ac8dac4c admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:33.395009 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cb0c36-6de7-467b-a36c-4075ac8dac4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:33.397267 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cb0c36-6de7-467b-a36c-4075ac8dac4c admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:33.421783 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1150/2329] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:43:33 2022] POST /identity/v3/users => generated 289 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:43:33.431367 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb0c36-6de7-467b-a36c-4075ac8dac4c admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:33.432125 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb7e63e-7c5b-4c8c-95f5-24ca86294939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:33.433876 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb7e63e-7c5b-4c8c-95f5-24ca86294939 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:33.434384 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb7e63e-7c5b-4c8c-95f5-24ca86294939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:33.434903 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb7e63e-7c5b-4c8c-95f5-24ca86294939 admin admin] PATH_INFO: `/v3/projects/e218e708fc4342eb9036d7ebd9350c13/users/bc26550fce3e43f091dd17f0fe65945c/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:33.459932 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb0c36-6de7-467b-a36c-4075ac8dac4c admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:33.464885 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f6c02-6e3a-4cda-9d8b-cb0319dda482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:33.465904 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f6c02-6e3a-4cda-9d8b-cb0319dda482 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:33.466348 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f6c02-6e3a-4cda-9d8b-cb0319dda482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:33.466712 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f6c02-6e3a-4cda-9d8b-cb0319dda482 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:33.467639 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321f6c02-6e3a-4cda-9d8b-cb0319dda482 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:33.469992 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321f6c02-6e3a-4cda-9d8b-cb0319dda482 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:33.478023 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb7e63e-7c5b-4c8c-95f5-24ca86294939 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e218e708fc4342eb9036d7ebd9350c13, user_id=bc26550fce3e43f091dd17f0fe65945c, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:33.482113 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1151/2330] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:43:33 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:43:33.487458 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f6c02-6e3a-4cda-9d8b-cb0319dda482 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:33.487786 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb7e63e-7c5b-4c8c-95f5-24ca86294939 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:33.499121 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f6c02-6e3a-4cda-9d8b-cb0319dda482 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:33.523070 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1180/2331] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:43:33 2022] PUT /identity/v3/projects/e218e708fc4342eb9036d7ebd9350c13/users/bc26550fce3e43f091dd17f0fe65945c/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:43:33.529178 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24728747-a721-47d1-876b-0b7e03eb34fd None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:33.529796 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24728747-a721-47d1-876b-0b7e03eb34fd None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:33.530421 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24728747-a721-47d1-876b-0b7e03eb34fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:33.586406 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ec3150-3e19-412f-b634-f10468cb0874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:33.588028 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec3150-3e19-412f-b634-f10468cb0874 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:33.588371 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec3150-3e19-412f-b634-f10468cb0874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:33.589786 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec3150-3e19-412f-b634-f10468cb0874 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:33.591668 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ec3150-3e19-412f-b634-f10468cb0874 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:33.595116 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ec3150-3e19-412f-b634-f10468cb0874 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:33.596649 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:33.613889 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1152/2332] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:43:33 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:43:33.615194 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ec3150-3e19-412f-b634-f10468cb0874 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:33.625584 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-24728747-a721-47d1-876b-0b7e03eb34fd None None] MFA Rules not processed for user `bc26550fce3e43f091dd17f0fe65945c`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:33.626984 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-24728747-a721-47d1-876b-0b7e03eb34fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:33.629562 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ec3150-3e19-412f-b634-f10468cb0874 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:33.636683 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e34521-ff5b-490d-aab1-f7e4bbbbad1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:33.638301 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e34521-ff5b-490d-aab1-f7e4bbbbad1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:33.638591 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e34521-ff5b-490d-aab1-f7e4bbbbad1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:33.638868 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e34521-ff5b-490d-aab1-f7e4bbbbad1c admin admin] PATH_INFO: `/v3/projects/5f146875b82145208428f5ea7de57943/users/0e30ae076f914d8fa9cf365fc32e981b/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:33.667030 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e34521-ff5b-490d-aab1-f7e4bbbbad1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f146875b82145208428f5ea7de57943, user_id=0e30ae076f914d8fa9cf365fc32e981b, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:33.669819 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e34521-ff5b-490d-aab1-f7e4bbbbad1c admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:33.708777 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1181/2333] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:43:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:43:33.712711 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1153/2334] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:43:33 2022] PUT /identity/v3/projects/5f146875b82145208428f5ea7de57943/users/0e30ae076f914d8fa9cf365fc32e981b/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:33.721507 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24728747-a721-47d1-876b-0b7e03eb34fd None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:33.723825 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-24728747-a721-47d1-876b-0b7e03eb34fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:33.727106 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc2c349-81b7-4648-bbed-6fa5e52b0a22 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:33.727535 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc2c349-81b7-4648-bbed-6fa5e52b0a22 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:33.727807 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc2c349-81b7-4648-bbed-6fa5e52b0a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:33.767209 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:33.773610 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-4fc2c349-81b7-4648-bbed-6fa5e52b0a22 None None] MFA Rules not processed for user `0e30ae076f914d8fa9cf365fc32e981b`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:33.780570 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-4fc2c349-81b7-4648-bbed-6fa5e52b0a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:33.833082 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24728747-a721-47d1-876b-0b7e03eb34fd None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:33.872944 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1154/2335] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:43:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:43:33.882918 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc2c349-81b7-4648-bbed-6fa5e52b0a22 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:33.886726 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-4fc2c349-81b7-4648-bbed-6fa5e52b0a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:33.902302 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f55fc11-fc02-48f5-b033-19ea26d3d672 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:33.902963 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f55fc11-fc02-48f5-b033-19ea26d3d672 service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:33.903248 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f55fc11-fc02-48f5-b033-19ea26d3d672 service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:33.903528 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f55fc11-fc02-48f5-b033-19ea26d3d672 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:33.975576 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f55fc11-fc02-48f5-b033-19ea26d3d672 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:33.977305 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f55fc11-fc02-48f5-b033-19ea26d3d672 service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:33.981451 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc2c349-81b7-4648-bbed-6fa5e52b0a22 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:33.988088 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4901c74a-ade6-4be4-afd0-5512ab588588 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:33.989048 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4901c74a-ade6-4be4-afd0-5512ab588588 service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:33.989528 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4901c74a-ade6-4be4-afd0-5512ab588588 service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:33.989671 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1182/2336] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:43:33 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:43:33.990205 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4901c74a-ade6-4be4-afd0-5512ab588588 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:34.003211 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4901c74a-ade6-4be4-afd0-5512ab588588 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:34.005394 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4901c74a-ade6-4be4-afd0-5512ab588588 service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:34.020248 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1155/2337] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:43:33 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:43:36.593374 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a6555-e629-4379-b12d-7c5cc81b7ae4 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:36.593960 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a6555-e629-4379-b12d-7c5cc81b7ae4 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:36.594661 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a6555-e629-4379-b12d-7c5cc81b7ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:36.614463 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-5d4a6555-e629-4379-b12d-7c5cc81b7ae4 None None] MFA Rules not processed for user `bf905723382e4372ae63d77a376982ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:36.615719 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-5d4a6555-e629-4379-b12d-7c5cc81b7ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:36.670776 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1183/2338] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:43:36 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:36.708975 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4901c74a-ade6-4be4-afd0-5512ab588588 service cinder] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:36.782687 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4901c74a-ade6-4be4-afd0-5512ab588588 service cinder] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:36.787398 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25710fa9-539d-4d0c-a94f-349d881619c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:36.788445 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25710fa9-539d-4d0c-a94f-349d881619c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:36.788736 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25710fa9-539d-4d0c-a94f-349d881619c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:36.789017 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25710fa9-539d-4d0c-a94f-349d881619c5 admin admin] PATH_INFO: `/v3/users/bc26550fce3e43f091dd17f0fe65945c` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:36.790268 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25710fa9-539d-4d0c-a94f-349d881619c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc26550fce3e43f091dd17f0fe65945c)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:36.792257 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25710fa9-539d-4d0c-a94f-349d881619c5 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:36.798679 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:36.848625 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-25710fa9-539d-4d0c-a94f-349d881619c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc26550fce3e43f091dd17f0fe65945c'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:36.864309 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-25710fa9-539d-4d0c-a94f-349d881619c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc26550fce3e43f091dd17f0fe65945c'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:36.865151 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-25710fa9-539d-4d0c-a94f-349d881619c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc26550fce3e43f091dd17f0fe65945c'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:36.875048 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-25710fa9-539d-4d0c-a94f-349d881619c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc26550fce3e43f091dd17f0fe65945c'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:36.875797 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-25710fa9-539d-4d0c-a94f-349d881619c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc26550fce3e43f091dd17f0fe65945c'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:36.890653 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1156/2339] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:43:36 2022] DELETE /identity/v3/users/bc26550fce3e43f091dd17f0fe65945c => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:36.895545 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8834d61-7be5-4b8a-bf11-6b5474f6d23b None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:36.895855 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8834d61-7be5-4b8a-bf11-6b5474f6d23b None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:36.896109 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8834d61-7be5-4b8a-bf11-6b5474f6d23b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:36.915249 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-e8834d61-7be5-4b8a-bf11-6b5474f6d23b None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:36.916496 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-e8834d61-7be5-4b8a-bf11-6b5474f6d23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:36.940939 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:36.994417 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1184/2340] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:43:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:43:37.004961 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25710fa9-539d-4d0c-a94f-349d881619c5 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:37.007017 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-25710fa9-539d-4d0c-a94f-349d881619c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:37.074045 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25710fa9-539d-4d0c-a94f-349d881619c5 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:37.077747 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0d69cb-c216-477f-9d8a-39b5cd56559b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:37.078618 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d69cb-c216-477f-9d8a-39b5cd56559b service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:37.079080 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d69cb-c216-477f-9d8a-39b5cd56559b service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:37.079363 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d69cb-c216-477f-9d8a-39b5cd56559b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:37.089439 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0d69cb-c216-477f-9d8a-39b5cd56559b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:37.090919 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0d69cb-c216-477f-9d8a-39b5cd56559b service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:37.102659 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1157/2341] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:43:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:43:37.402156 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8834d61-7be5-4b8a-bf11-6b5474f6d23b None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:37.403924 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-e8834d61-7be5-4b8a-bf11-6b5474f6d23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:37.420482 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8834d61-7be5-4b8a-bf11-6b5474f6d23b None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:37.426226 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127734c3-9785-4a98-b441-9061ce25f2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:37.427111 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127734c3-9785-4a98-b441-9061ce25f2f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:37.427573 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127734c3-9785-4a98-b441-9061ce25f2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:37.427860 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127734c3-9785-4a98-b441-9061ce25f2f3 admin admin] PATH_INFO: `/v3/projects/e218e708fc4342eb9036d7ebd9350c13` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:37.429165 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127734c3-9785-4a98-b441-9061ce25f2f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e218e708fc4342eb9036d7ebd9350c13)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:37.438683 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127734c3-9785-4a98-b441-9061ce25f2f3 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:37.496329 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-127734c3-9785-4a98-b441-9061ce25f2f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e218e708fc4342eb9036d7ebd9350c13', 'request_id': 'req-127734c3-9785-4a98-b441-9061ce25f2f3'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:37.503963 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-127734c3-9785-4a98-b441-9061ce25f2f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e218e708fc4342eb9036d7ebd9350c13', 'request_id': 'req-127734c3-9785-4a98-b441-9061ce25f2f3'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:37.512122 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-127734c3-9785-4a98-b441-9061ce25f2f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e218e708fc4342eb9036d7ebd9350c13', 'request_id': 'req-127734c3-9785-4a98-b441-9061ce25f2f3'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:37.529108 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-127734c3-9785-4a98-b441-9061ce25f2f3 admin admin] The token cache is being invalidate because project e218e708fc4342eb9036d7ebd9350c13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:43:37.529496 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-127734c3-9785-4a98-b441-9061ce25f2f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:37.530528 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-127734c3-9785-4a98-b441-9061ce25f2f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:37.533611 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1185/2342] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:43:37 2022] DELETE /identity/v3/projects/e218e708fc4342eb9036d7ebd9350c13 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:37.538927 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564b4dd-a742-4632-9fb5-7305d1e7e8d9 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:37.539242 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564b4dd-a742-4632-9fb5-7305d1e7e8d9 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:37.539536 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564b4dd-a742-4632-9fb5-7305d1e7e8d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:37.566207 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-2564b4dd-a742-4632-9fb5-7305d1e7e8d9 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:37.567555 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-2564b4dd-a742-4632-9fb5-7305d1e7e8d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:37.661141 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1158/2343] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:43:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:37.668655 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e0327-15e1-4784-8b8e-85b9eafdb7f9 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:37.669138 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e0327-15e1-4784-8b8e-85b9eafdb7f9 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:37.669245 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e0327-15e1-4784-8b8e-85b9eafdb7f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:37.692855 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-6a4e0327-15e1-4784-8b8e-85b9eafdb7f9 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:37.693280 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-6a4e0327-15e1-4784-8b8e-85b9eafdb7f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:37.702176 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1186/2344] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:43:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:37.707281 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2564b4dd-a742-4632-9fb5-7305d1e7e8d9 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:37.717476 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2564b4dd-a742-4632-9fb5-7305d1e7e8d9 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:37.722635 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3d5767-a9ce-493a-a6f9-79a055bb4ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:37.723320 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d5767-a9ce-493a-a6f9-79a055bb4ff7 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:37.723652 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d5767-a9ce-493a-a6f9-79a055bb4ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:37.723937 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d5767-a9ce-493a-a6f9-79a055bb4ff7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:37.724677 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3d5767-a9ce-493a-a6f9-79a055bb4ff7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:43:37.725032 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3d5767-a9ce-493a-a6f9-79a055bb4ff7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:37.726784 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3d5767-a9ce-493a-a6f9-79a055bb4ff7 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:37.739750 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1159/2345] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:43:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:43:37.744339 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4e0327-15e1-4784-8b8e-85b9eafdb7f9 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:37.755516 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4e0327-15e1-4784-8b8e-85b9eafdb7f9 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:37.761418 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a92ff13-9f4a-4c4a-89e8-70e9114a15d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:37.762198 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a92ff13-9f4a-4c4a-89e8-70e9114a15d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:37.762388 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a92ff13-9f4a-4c4a-89e8-70e9114a15d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:37.762677 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a92ff13-9f4a-4c4a-89e8-70e9114a15d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:37.763383 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a92ff13-9f4a-4c4a-89e8-70e9114a15d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:37.766275 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a92ff13-9f4a-4c4a-89e8-70e9114a15d8 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:37.801715 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1187/2346] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:43:37 2022] POST /identity/v3/projects => generated 379 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:43:37.805045 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3d5767-a9ce-493a-a6f9-79a055bb4ff7 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:37.816209 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3d5767-a9ce-493a-a6f9-79a055bb4ff7 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:37.826248 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:37.881875 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff83627c-87c4-49d9-ab0b-a14afe702ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:37.882554 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff83627c-87c4-49d9-ab0b-a14afe702ba4 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:37.882840 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff83627c-87c4-49d9-ab0b-a14afe702ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:37.883118 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff83627c-87c4-49d9-ab0b-a14afe702ba4 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:37.883809 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff83627c-87c4-49d9-ab0b-a14afe702ba4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:37.885828 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff83627c-87c4-49d9-ab0b-a14afe702ba4 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:37.910872 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1160/2347] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:43:37 2022] POST /identity/v3/users => generated 290 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:43:37.912701 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a92ff13-9f4a-4c4a-89e8-70e9114a15d8 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:37.926678 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a92ff13-9f4a-4c4a-89e8-70e9114a15d8 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:37.932391 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf487ec-5b12-4638-b16c-a59c1ec76cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:37.933063 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf487ec-5b12-4638-b16c-a59c1ec76cb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:37.933371 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf487ec-5b12-4638-b16c-a59c1ec76cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:37.933631 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf487ec-5b12-4638-b16c-a59c1ec76cb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:37.934736 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf487ec-5b12-4638-b16c-a59c1ec76cb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:37.936938 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf487ec-5b12-4638-b16c-a59c1ec76cb3 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:37.951259 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1188/2348] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:43:37 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:43:37.955697 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff83627c-87c4-49d9-ab0b-a14afe702ba4 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:37.966226 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff83627c-87c4-49d9-ab0b-a14afe702ba4 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:37.971558 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760e1b32-b548-418f-bc28-aecb02427bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:37.972250 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760e1b32-b548-418f-bc28-aecb02427bcf admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:37.972537 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760e1b32-b548-418f-bc28-aecb02427bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:37.972817 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760e1b32-b548-418f-bc28-aecb02427bcf admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:37.973542 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760e1b32-b548-418f-bc28-aecb02427bcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:37.975179 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760e1b32-b548-418f-bc28-aecb02427bcf admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:37.985403 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1161/2349] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:43:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:43:37.989812 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf487ec-5b12-4638-b16c-a59c1ec76cb3 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:38.000609 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf487ec-5b12-4638-b16c-a59c1ec76cb3 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:38.005540 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0529d5c-0e2d-49bb-9b9f-7b42d64db82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:38.006549 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0529d5c-0e2d-49bb-9b9f-7b42d64db82a admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:38.006937 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0529d5c-0e2d-49bb-9b9f-7b42d64db82a admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:38.007232 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0529d5c-0e2d-49bb-9b9f-7b42d64db82a admin admin] PATH_INFO: `/v3/projects/84c244fc43f94ea2bc9f76965c111e78/users/2039bddb840c4ac784f0b1c52326c79f/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:38.028087 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0529d5c-0e2d-49bb-9b9f-7b42d64db82a admin admin] RBAC: Authorizing `identity:create_grant(project_id=84c244fc43f94ea2bc9f76965c111e78, user_id=2039bddb840c4ac784f0b1c52326c79f, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:38.032128 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0529d5c-0e2d-49bb-9b9f-7b42d64db82a admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:38.045705 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1189/2350] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:43:37 2022] PUT /identity/v3/projects/84c244fc43f94ea2bc9f76965c111e78/users/2039bddb840c4ac784f0b1c52326c79f/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:38.050946 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b573a94-956b-43b9-b35b-d63f828f1e7d None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:38.051266 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b573a94-956b-43b9-b35b-d63f828f1e7d None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:38.051549 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b573a94-956b-43b9-b35b-d63f828f1e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:38.070222 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-1b573a94-956b-43b9-b35b-d63f828f1e7d None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:38.071441 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-1b573a94-956b-43b9-b35b-d63f828f1e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:38.134079 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1162/2351] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:43:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:38.144320 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0529d5c-0e2d-49bb-9b9f-7b42d64db82a admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:38.146108 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b0529d5c-0e2d-49bb-9b9f-7b42d64db82a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:38.193109 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:38.208346 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0529d5c-0e2d-49bb-9b9f-7b42d64db82a admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:38.212353 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53873f6-3ecd-4b60-be76-5cde07c06023 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:38.213011 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53873f6-3ecd-4b60-be76-5cde07c06023 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:38.213298 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53873f6-3ecd-4b60-be76-5cde07c06023 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:38.213577 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53873f6-3ecd-4b60-be76-5cde07c06023 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:38.224562 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53873f6-3ecd-4b60-be76-5cde07c06023 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:38.226057 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53873f6-3ecd-4b60-be76-5cde07c06023 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:38.236562 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1190/2352] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:43:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:43:39.229856 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6ce09-f5d4-4eb4-a17d-3b492c61fc4e None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:39.230363 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6ce09-f5d4-4eb4-a17d-3b492c61fc4e None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:39.230485 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6ce09-f5d4-4eb4-a17d-3b492c61fc4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:39.260197 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-7ec6ce09-f5d4-4eb4-a17d-3b492c61fc4e None None] MFA Rules not processed for user `2039bddb840c4ac784f0b1c52326c79f`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:39.261789 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-7ec6ce09-f5d4-4eb4-a17d-3b492c61fc4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:39.340739 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1163/2353] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:43:39 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:43:39.352054 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53873f6-3ecd-4b60-be76-5cde07c06023 service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:39.354033 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-f53873f6-3ecd-4b60-be76-5cde07c06023 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:39.416634 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53873f6-3ecd-4b60-be76-5cde07c06023 service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:39.421380 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39e378d-8f74-4525-9bfa-02beea2f81d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:39.422743 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e378d-8f74-4525-9bfa-02beea2f81d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:39.422743 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e378d-8f74-4525-9bfa-02beea2f81d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:39.422743 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e378d-8f74-4525-9bfa-02beea2f81d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:39.438578 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39e378d-8f74-4525-9bfa-02beea2f81d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:39.439818 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39e378d-8f74-4525-9bfa-02beea2f81d3 service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:39.450647 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1191/2354] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:43:39 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:43:40.827004 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b0874c-f914-4dac-82f6-830d63f6aceb None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:40.827498 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b0874c-f914-4dac-82f6-830d63f6aceb None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:40.827710 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b0874c-f914-4dac-82f6-830d63f6aceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:40.845767 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-d2b0874c-f914-4dac-82f6-830d63f6aceb None None] MFA Rules not processed for user `2039bddb840c4ac784f0b1c52326c79f`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:40.846964 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d2b0874c-f914-4dac-82f6-830d63f6aceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:40.853797 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1164/2355] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:43:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:43.376266 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39e378d-8f74-4525-9bfa-02beea2f81d3 service cinder] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:43.380540 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-a39e378d-8f74-4525-9bfa-02beea2f81d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:43.383803 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b0874c-f914-4dac-82f6-830d63f6aceb None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:43.385532 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d2b0874c-f914-4dac-82f6-830d63f6aceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:43.464670 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:43.528817 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39e378d-8f74-4525-9bfa-02beea2f81d3 service cinder] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:43.536153 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fbbf54-f718-44df-ab85-31467aa5ac62 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39cc268bad604dd08594e3b39c7ecf5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:43.537084 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fbbf54-f718-44df-ab85-31467aa5ac62 service placement] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:43.537372 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fbbf54-f718-44df-ab85-31467aa5ac62 service placement] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:43.537648 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fbbf54-f718-44df-ab85-31467aa5ac62 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:43.540431 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-36fbbf54-f718-44df-ab85-31467aa5ac62 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:43.559178 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fbbf54-f718-44df-ab85-31467aa5ac62 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:43.561203 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fbbf54-f718-44df-ab85-31467aa5ac62 service placement] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:43.598738 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b0874c-f914-4dac-82f6-830d63f6aceb None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:43.599379 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1192/2356] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:43:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:43:43.625098 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b14acb2-4db3-4b42-951f-3c1f8ea766b5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39cc268bad604dd08594e3b39c7ecf5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:43.625883 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b14acb2-4db3-4b42-951f-3c1f8ea766b5 service placement] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:43.627394 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b14acb2-4db3-4b42-951f-3c1f8ea766b5 service placement] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:43.627697 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b14acb2-4db3-4b42-951f-3c1f8ea766b5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:43.642809 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b14acb2-4db3-4b42-951f-3c1f8ea766b5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:43.645187 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b14acb2-4db3-4b42-951f-3c1f8ea766b5 service placement] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:43.666277 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1165/2357] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:43:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:43:45.080596 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b8258-51fa-4e4f-827e-292b148649e5 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:45.081133 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b8258-51fa-4e4f-827e-292b148649e5 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:45.081246 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b8258-51fa-4e4f-827e-292b148649e5 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:45.082676 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1193/2358] 10.222.0.13 () {54 vars in 921 bytes} [Fri Jun 24 08:43:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:43:45.088882 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56781d09-eb0f-47c1-961c-a1e1fe79d78c None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:45.089359 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56781d09-eb0f-47c1-961c-a1e1fe79d78c None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:45.089589 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56781d09-eb0f-47c1-961c-a1e1fe79d78c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:45.167724 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-56781d09-eb0f-47c1-961c-a1e1fe79d78c None None] MFA Rules not processed for user `74bb1bc6821b4500b26f160ccae5b7d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:45.169133 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-56781d09-eb0f-47c1-961c-a1e1fe79d78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:45.256846 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1166/2359] 10.222.0.13 () {58 vars in 1018 bytes} [Fri Jun 24 08:43:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:45.276737 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03b8258-51fa-4e4f-827e-292b148649e5 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:45.278494 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-d03b8258-51fa-4e4f-827e-292b148649e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:45.354335 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03b8258-51fa-4e4f-827e-292b148649e5 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:45.358043 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b2e207-7d06-4357-8270-cba546cc7d6f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd15c5f6aeb874191b73c8ee0410c6bc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:45.359601 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b2e207-7d06-4357-8270-cba546cc7d6f service glance] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:45.360565 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b2e207-7d06-4357-8270-cba546cc7d6f service glance] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:45.360855 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b2e207-7d06-4357-8270-cba546cc7d6f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:45.372030 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b2e207-7d06-4357-8270-cba546cc7d6f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:45.373824 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b2e207-7d06-4357-8270-cba546cc7d6f service glance] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:45.381228 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:45.389634 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1194/2360] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:43:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:43:45.462714 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac0b75-e993-45d7-95dd-6ab9a97589b0 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:45.463009 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac0b75-e993-45d7-95dd-6ab9a97589b0 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:45.463292 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac0b75-e993-45d7-95dd-6ab9a97589b0 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:45.464732 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1167/2361] 10.222.0.13 () {54 vars in 921 bytes} [Fri Jun 24 08:43:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:43:45.471677 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981fc9e7-97db-42b7-966b-579c6d4dc5cb None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:45.471977 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981fc9e7-97db-42b7-966b-579c6d4dc5cb None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:45.472262 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981fc9e7-97db-42b7-966b-579c6d4dc5cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:45.493712 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-981fc9e7-97db-42b7-966b-579c6d4dc5cb None None] MFA Rules not processed for user `c5f7ba6c7faf43e7acbb43eee58d8c8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:45.495003 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-981fc9e7-97db-42b7-966b-579c6d4dc5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:45.503166 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1195/2362] 10.222.0.13 () {58 vars in 1018 bytes} [Fri Jun 24 08:43:45 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:45.516314 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac0b75-e993-45d7-95dd-6ab9a97589b0 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:45.518162 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-a8ac0b75-e993-45d7-95dd-6ab9a97589b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:45.532934 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac0b75-e993-45d7-95dd-6ab9a97589b0 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:45.536663 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16581bab-1ad4-46d1-9adb-a33a784ab452 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:45.537324 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16581bab-1ad4-46d1-9adb-a33a784ab452 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:45.537634 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16581bab-1ad4-46d1-9adb-a33a784ab452 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:45.538565 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16581bab-1ad4-46d1-9adb-a33a784ab452 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:45.548566 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16581bab-1ad4-46d1-9adb-a33a784ab452 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:45.550767 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16581bab-1ad4-46d1-9adb-a33a784ab452 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:45.566526 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1168/2363] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:43:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:43:45.669251 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafe006-bb70-466c-bc24-cf823acb8183 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:45.669547 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafe006-bb70-466c-bc24-cf823acb8183 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:45.669869 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafe006-bb70-466c-bc24-cf823acb8183 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:45.672323 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1196/2364] 10.222.0.13 () {54 vars in 921 bytes} [Fri Jun 24 08:43:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:43:45.677507 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd7fdf5-ea65-4287-97c4-2aedbfb7c766 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:45.678025 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd7fdf5-ea65-4287-97c4-2aedbfb7c766 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:45.678303 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd7fdf5-ea65-4287-97c4-2aedbfb7c766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:45.699999 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-7fd7fdf5-ea65-4287-97c4-2aedbfb7c766 None None] MFA Rules not processed for user `74bb1bc6821b4500b26f160ccae5b7d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:45.701415 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-7fd7fdf5-ea65-4287-97c4-2aedbfb7c766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:45.751036 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1169/2365] 10.222.0.13 () {58 vars in 1018 bytes} [Fri Jun 24 08:43:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:45.766799 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dafe006-bb70-466c-bc24-cf823acb8183 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:45.788298 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dafe006-bb70-466c-bc24-cf823acb8183 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:45.793235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cfd308-15da-4c70-988b-69a6067f0c36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:45.794163 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cfd308-15da-4c70-988b-69a6067f0c36 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:45.794456 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cfd308-15da-4c70-988b-69a6067f0c36 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:45.794737 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cfd308-15da-4c70-988b-69a6067f0c36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:45.809339 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cfd308-15da-4c70-988b-69a6067f0c36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:45.810940 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cfd308-15da-4c70-988b-69a6067f0c36 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:45.824253 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1197/2366] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:43:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:43:49.683396 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f2820-5ce1-4c7c-a8be-89c7944d25bc None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:49.683934 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f2820-5ce1-4c7c-a8be-89c7944d25bc None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:49.683934 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f2820-5ce1-4c7c-a8be-89c7944d25bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:49.701684 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-d94f2820-5ce1-4c7c-a8be-89c7944d25bc None None] MFA Rules not processed for user `71e0fd73e5c9430c813af1de9ccd4a52`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:49.703288 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d94f2820-5ce1-4c7c-a8be-89c7944d25bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:49.733429 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:49.763297 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1170/2367] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:43:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:54.357814 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebbc1f4-80e1-4991-a6df-5ad6c4816d7d None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:54.360875 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebbc1f4-80e1-4991-a6df-5ad6c4816d7d None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:54.361349 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebbc1f4-80e1-4991-a6df-5ad6c4816d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:54.489649 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-1ebbc1f4-80e1-4991-a6df-5ad6c4816d7d None None] MFA Rules not processed for user `85d9c4c57681447c898ebfd7990b05b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:54.491899 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-1ebbc1f4-80e1-4991-a6df-5ad6c4816d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:54.587717 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1198/2368] 10.222.0.13 () {58 vars in 1009 bytes} [Fri Jun 24 08:43:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:54.618011 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f2820-5ce1-4c7c-a8be-89c7944d25bc None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:54.620358 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d94f2820-5ce1-4c7c-a8be-89c7944d25bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:54.741952 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f2820-5ce1-4c7c-a8be-89c7944d25bc None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:54.747622 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f119f732-a077-4f7a-8028-1bc27d7b40e1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec8c2f113cf41a0939b81f38fbee813', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:54.748627 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119f732-a077-4f7a-8028-1bc27d7b40e1 service swift] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:54.748964 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119f732-a077-4f7a-8028-1bc27d7b40e1 service swift] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:54.749961 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119f732-a077-4f7a-8028-1bc27d7b40e1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:54.760219 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f119f732-a077-4f7a-8028-1bc27d7b40e1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:54.762574 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f119f732-a077-4f7a-8028-1bc27d7b40e1 service swift] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:54.771945 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1171/2369] 10.222.0.13 () {60 vars in 1384 bytes} [Fri Jun 24 08:43:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 08:43:55.611362 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebbc1f4-80e1-4991-a6df-5ad6c4816d7d None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:55.612821 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-1ebbc1f4-80e1-4991-a6df-5ad6c4816d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:55.634601 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebbc1f4-80e1-4991-a6df-5ad6c4816d7d None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:55.641575 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf1e70-860f-4317-9ea7-8baa7cfe3781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:55.642764 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf1e70-860f-4317-9ea7-8baa7cfe3781 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:55.643053 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf1e70-860f-4317-9ea7-8baa7cfe3781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:55.643460 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf1e70-860f-4317-9ea7-8baa7cfe3781 admin admin] PATH_INFO: `/v3/users/0e30ae076f914d8fa9cf365fc32e981b` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:55.644889 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bf1e70-860f-4317-9ea7-8baa7cfe3781 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e30ae076f914d8fa9cf365fc32e981b)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:55.647362 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bf1e70-860f-4317-9ea7-8baa7cfe3781 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:55.706788 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-06bf1e70-860f-4317-9ea7-8baa7cfe3781 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e30ae076f914d8fa9cf365fc32e981b'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:55.732098 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-06bf1e70-860f-4317-9ea7-8baa7cfe3781 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e30ae076f914d8fa9cf365fc32e981b'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:55.741727 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-06bf1e70-860f-4317-9ea7-8baa7cfe3781 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e30ae076f914d8fa9cf365fc32e981b'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:55.742684 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-06bf1e70-860f-4317-9ea7-8baa7cfe3781 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e30ae076f914d8fa9cf365fc32e981b'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:55.743267 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-06bf1e70-860f-4317-9ea7-8baa7cfe3781 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e30ae076f914d8fa9cf365fc32e981b'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:55.760816 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1199/2370] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:43:55 2022] DELETE /identity/v3/users/0e30ae076f914d8fa9cf365fc32e981b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:55.765913 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523abe4-d7b7-4279-ab51-62244176526e None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:55.766433 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523abe4-d7b7-4279-ab51-62244176526e None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:55.766433 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523abe4-d7b7-4279-ab51-62244176526e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:55.786349 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-5523abe4-d7b7-4279-ab51-62244176526e None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:55.788492 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-5523abe4-d7b7-4279-ab51-62244176526e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:55.903671 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1172/2371] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:43:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:43:55.913740 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf1e70-860f-4317-9ea7-8baa7cfe3781 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:55.915194 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-06bf1e70-860f-4317-9ea7-8baa7cfe3781 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:55.957770 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:56.015401 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf1e70-860f-4317-9ea7-8baa7cfe3781 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:56.024281 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de748d8f-55ad-4901-ad53-68a020560a00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:56.024945 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de748d8f-55ad-4901-ad53-68a020560a00 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:56.025226 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de748d8f-55ad-4901-ad53-68a020560a00 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:56.025499 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de748d8f-55ad-4901-ad53-68a020560a00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:56.035966 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de748d8f-55ad-4901-ad53-68a020560a00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:56.037410 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de748d8f-55ad-4901-ad53-68a020560a00 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:56.048935 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1200/2372] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:43:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:43:56.507037 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5523abe4-d7b7-4279-ab51-62244176526e None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:56.516114 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-5523abe4-d7b7-4279-ab51-62244176526e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:56.532401 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5523abe4-d7b7-4279-ab51-62244176526e None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:56.558699 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ef6359-266f-4d6f-8f8c-674fdcf835cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:56.560124 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef6359-266f-4d6f-8f8c-674fdcf835cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:56.562861 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef6359-266f-4d6f-8f8c-674fdcf835cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:56.563183 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef6359-266f-4d6f-8f8c-674fdcf835cf admin admin] PATH_INFO: `/v3/projects/5f146875b82145208428f5ea7de57943` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:56.565078 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ef6359-266f-4d6f-8f8c-674fdcf835cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f146875b82145208428f5ea7de57943)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:56.568865 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ef6359-266f-4d6f-8f8c-674fdcf835cf admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:56.683828 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:56.691618 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f8ef6359-266f-4d6f-8f8c-674fdcf835cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f146875b82145208428f5ea7de57943', 'request_id': 'req-f8ef6359-266f-4d6f-8f8c-674fdcf835cf'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:56.701631 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f8ef6359-266f-4d6f-8f8c-674fdcf835cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f146875b82145208428f5ea7de57943', 'request_id': 'req-f8ef6359-266f-4d6f-8f8c-674fdcf835cf'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:56.713910 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f8ef6359-266f-4d6f-8f8c-674fdcf835cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f146875b82145208428f5ea7de57943', 'request_id': 'req-f8ef6359-266f-4d6f-8f8c-674fdcf835cf'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:56.743651 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f8ef6359-266f-4d6f-8f8c-674fdcf835cf admin admin] The token cache is being invalidate because project 5f146875b82145208428f5ea7de57943 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:43:56.744163 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f8ef6359-266f-4d6f-8f8c-674fdcf835cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:56.744868 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f8ef6359-266f-4d6f-8f8c-674fdcf835cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:56.750498 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1173/2373] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:43:56 2022] DELETE /identity/v3/projects/5f146875b82145208428f5ea7de57943 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:43:56.752276 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e264063-188c-44ab-a978-4bc7f4101687 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:56.752767 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e264063-188c-44ab-a978-4bc7f4101687 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:56.752914 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e264063-188c-44ab-a978-4bc7f4101687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:56.790254 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-2e264063-188c-44ab-a978-4bc7f4101687 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:56.794168 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-2e264063-188c-44ab-a978-4bc7f4101687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:56.906920 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1201/2374] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:43:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:56.910418 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea0c4c-93dc-4ea8-b6ee-4637cf1df942 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:56.911748 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea0c4c-93dc-4ea8-b6ee-4637cf1df942 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:56.912274 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea0c4c-93dc-4ea8-b6ee-4637cf1df942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:56.935215 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-09ea0c4c-93dc-4ea8-b6ee-4637cf1df942 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:56.936929 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-09ea0c4c-93dc-4ea8-b6ee-4637cf1df942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:56.951744 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1174/2375] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:43:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:56.952206 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e264063-188c-44ab-a978-4bc7f4101687 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:56.966596 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e264063-188c-44ab-a978-4bc7f4101687 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:56.975207 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8b43f7-738f-4910-bbbb-652c695d93eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:56.975991 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b43f7-738f-4910-bbbb-652c695d93eb admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:56.976345 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b43f7-738f-4910-bbbb-652c695d93eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:56.976693 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b43f7-738f-4910-bbbb-652c695d93eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:56.977502 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8b43f7-738f-4910-bbbb-652c695d93eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:43:56.977922 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8b43f7-738f-4910-bbbb-652c695d93eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:56.979639 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8b43f7-738f-4910-bbbb-652c695d93eb admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:57.005288 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1202/2376] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:43:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:43:57.008584 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea0c4c-93dc-4ea8-b6ee-4637cf1df942 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:57.021400 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea0c4c-93dc-4ea8-b6ee-4637cf1df942 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:57.028246 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b77d3-8e33-4f9f-a3e9-65c7f45eaf15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:57.028246 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b77d3-8e33-4f9f-a3e9-65c7f45eaf15 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:57.028246 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b77d3-8e33-4f9f-a3e9-65c7f45eaf15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:57.028246 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b77d3-8e33-4f9f-a3e9-65c7f45eaf15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:57.028684 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3b77d3-8e33-4f9f-a3e9-65c7f45eaf15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:57.031243 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3b77d3-8e33-4f9f-a3e9-65c7f45eaf15 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:57.075634 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8b43f7-738f-4910-bbbb-652c695d93eb admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:57.076562 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1175/2377] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:43:57 2022] POST /identity/v3/projects => generated 375 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:43:57.091001 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8b43f7-738f-4910-bbbb-652c695d93eb admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:57.199296 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fa8432-1abc-40ac-9e8e-e513624ad039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:57.200124 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa8432-1abc-40ac-9e8e-e513624ad039 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:57.200420 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa8432-1abc-40ac-9e8e-e513624ad039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:57.200707 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa8432-1abc-40ac-9e8e-e513624ad039 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:57.201439 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fa8432-1abc-40ac-9e8e-e513624ad039 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:57.204449 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fa8432-1abc-40ac-9e8e-e513624ad039 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:57.231399 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1203/2378] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:43:57 2022] POST /identity/v3/users => generated 288 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:43:57.235555 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b77d3-8e33-4f9f-a3e9-65c7f45eaf15 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:57.255279 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b77d3-8e33-4f9f-a3e9-65c7f45eaf15 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:57.265563 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1d571-5ee6-4609-ab36-23d847e4dd96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:57.266665 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1d571-5ee6-4609-ab36-23d847e4dd96 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:57.267145 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1d571-5ee6-4609-ab36-23d847e4dd96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:57.267645 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1d571-5ee6-4609-ab36-23d847e4dd96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:57.269174 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c1d571-5ee6-4609-ab36-23d847e4dd96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:57.271619 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c1d571-5ee6-4609-ab36-23d847e4dd96 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:57.287782 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1176/2379] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:43:57 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:43:57.292168 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fa8432-1abc-40ac-9e8e-e513624ad039 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:57.315871 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fa8432-1abc-40ac-9e8e-e513624ad039 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:57.321257 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792378ae-c19b-412a-940d-69cf7b633b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:57.322259 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792378ae-c19b-412a-940d-69cf7b633b04 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:57.322608 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792378ae-c19b-412a-940d-69cf7b633b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:57.322947 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792378ae-c19b-412a-940d-69cf7b633b04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:57.323766 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792378ae-c19b-412a-940d-69cf7b633b04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:57.325804 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792378ae-c19b-412a-940d-69cf7b633b04 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:57.341720 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1204/2380] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:43:57 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:43:57.345532 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1d571-5ee6-4609-ab36-23d847e4dd96 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:57.360045 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1d571-5ee6-4609-ab36-23d847e4dd96 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:57.365773 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3ddee2-a3f2-4187-8ff6-d49788c2ae96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:57.366697 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ddee2-a3f2-4187-8ff6-d49788c2ae96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:57.366970 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ddee2-a3f2-4187-8ff6-d49788c2ae96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:57.367247 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ddee2-a3f2-4187-8ff6-d49788c2ae96 admin admin] PATH_INFO: `/v3/projects/cb01c37604464a0abefda50c0c9d7d51/users/3357282e0362461f8b140bddbc6dc634/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:57.400176 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3ddee2-a3f2-4187-8ff6-d49788c2ae96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb01c37604464a0abefda50c0c9d7d51, user_id=3357282e0362461f8b140bddbc6dc634, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:57.400176 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3ddee2-a3f2-4187-8ff6-d49788c2ae96 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:57.416976 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1177/2381] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:43:57 2022] PUT /identity/v3/projects/cb01c37604464a0abefda50c0c9d7d51/users/3357282e0362461f8b140bddbc6dc634/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:57.427952 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa7ee6-b037-4cc1-a9b4-3018db5f184c None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:57.428247 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa7ee6-b037-4cc1-a9b4-3018db5f184c None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:57.428544 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa7ee6-b037-4cc1-a9b4-3018db5f184c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:57.463056 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-d9aa7ee6-b037-4cc1-a9b4-3018db5f184c None None] MFA Rules not processed for user `3357282e0362461f8b140bddbc6dc634`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:57.464420 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-d9aa7ee6-b037-4cc1-a9b4-3018db5f184c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:57.487530 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:57.551669 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1205/2382] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:43:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:57.565365 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3ddee2-a3f2-4187-8ff6-d49788c2ae96 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:57.567276 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-ee3ddee2-a3f2-4187-8ff6-d49788c2ae96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:57.682496 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3ddee2-a3f2-4187-8ff6-d49788c2ae96 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:57.690340 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2af455-6222-4692-a2e1-7fa1e55ccaba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:57.691062 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2af455-6222-4692-a2e1-7fa1e55ccaba service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:57.691518 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2af455-6222-4692-a2e1-7fa1e55ccaba service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:57.691871 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2af455-6222-4692-a2e1-7fa1e55ccaba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:57.702218 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2af455-6222-4692-a2e1-7fa1e55ccaba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:57.704396 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2af455-6222-4692-a2e1-7fa1e55ccaba service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:57.720228 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1178/2383] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:43:57 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:44:01.213495 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aa7ee6-b037-4cc1-a9b4-3018db5f184c None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:01.384643 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aa7ee6-b037-4cc1-a9b4-3018db5f184c None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:01.389891 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9dc247-5bdb-46e8-a304-71517a6f10be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:01.391318 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9dc247-5bdb-46e8-a304-71517a6f10be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:01.391588 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9dc247-5bdb-46e8-a304-71517a6f10be admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:01.391961 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9dc247-5bdb-46e8-a304-71517a6f10be admin admin] PATH_INFO: `/v3/users/3357282e0362461f8b140bddbc6dc634` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:01.395448 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9dc247-5bdb-46e8-a304-71517a6f10be admin admin] RBAC: Authorizing `identity:delete_user(user_id=3357282e0362461f8b140bddbc6dc634)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:01.405732 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9dc247-5bdb-46e8-a304-71517a6f10be admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:01.499592 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7e9dc247-5bdb-46e8-a304-71517a6f10be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3357282e0362461f8b140bddbc6dc634'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:01.526964 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7e9dc247-5bdb-46e8-a304-71517a6f10be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3357282e0362461f8b140bddbc6dc634'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:01.600352 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7e9dc247-5bdb-46e8-a304-71517a6f10be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3357282e0362461f8b140bddbc6dc634'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:01.600352 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7e9dc247-5bdb-46e8-a304-71517a6f10be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3357282e0362461f8b140bddbc6dc634'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:01.600352 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7e9dc247-5bdb-46e8-a304-71517a6f10be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3357282e0362461f8b140bddbc6dc634'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:01.662158 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1206/2384] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:44:01 2022] DELETE /identity/v3/users/3357282e0362461f8b140bddbc6dc634 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:44:01.670844 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2796e4e-ddc7-426b-8f1a-eae7358ccd5f None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:01.672532 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2796e4e-ddc7-426b-8f1a-eae7358ccd5f None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:01.672892 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2796e4e-ddc7-426b-8f1a-eae7358ccd5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:01.717748 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-d2796e4e-ddc7-426b-8f1a-eae7358ccd5f None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:44:01.721080 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d2796e4e-ddc7-426b-8f1a-eae7358ccd5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:01.855959 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1179/2385] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:44:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:44:01.867026 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9dc247-5bdb-46e8-a304-71517a6f10be admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:01.868786 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-7e9dc247-5bdb-46e8-a304-71517a6f10be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:01.961297 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:44:02.017222 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9dc247-5bdb-46e8-a304-71517a6f10be admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:02.029226 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcc0606-aada-421e-9b0c-2035b09e8e01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:02.030211 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc0606-aada-421e-9b0c-2035b09e8e01 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:02.030791 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc0606-aada-421e-9b0c-2035b09e8e01 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:02.031140 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc0606-aada-421e-9b0c-2035b09e8e01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:02.046529 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcc0606-aada-421e-9b0c-2035b09e8e01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:02.047354 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcc0606-aada-421e-9b0c-2035b09e8e01 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:02.088328 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1207/2386] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:44:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:44:02.440955 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2796e4e-ddc7-426b-8f1a-eae7358ccd5f None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:02.444197 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d2796e4e-ddc7-426b-8f1a-eae7358ccd5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:02.460743 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2796e4e-ddc7-426b-8f1a-eae7358ccd5f None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:02.465987 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7ff76e-a86f-4619-9455-9efd1b569f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:02.466819 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ff76e-a86f-4619-9455-9efd1b569f67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:02.467108 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ff76e-a86f-4619-9455-9efd1b569f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:02.467395 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ff76e-a86f-4619-9455-9efd1b569f67 admin admin] PATH_INFO: `/v3/projects/cb01c37604464a0abefda50c0c9d7d51` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:02.470096 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7ff76e-a86f-4619-9455-9efd1b569f67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb01c37604464a0abefda50c0c9d7d51)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:02.472060 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7ff76e-a86f-4619-9455-9efd1b569f67 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:02.485402 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:44:02.537205 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-9c7ff76e-a86f-4619-9455-9efd1b569f67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb01c37604464a0abefda50c0c9d7d51', 'request_id': 'req-9c7ff76e-a86f-4619-9455-9efd1b569f67'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:02.544125 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-9c7ff76e-a86f-4619-9455-9efd1b569f67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb01c37604464a0abefda50c0c9d7d51', 'request_id': 'req-9c7ff76e-a86f-4619-9455-9efd1b569f67'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:02.552969 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-9c7ff76e-a86f-4619-9455-9efd1b569f67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb01c37604464a0abefda50c0c9d7d51', 'request_id': 'req-9c7ff76e-a86f-4619-9455-9efd1b569f67'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:02.572629 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-9c7ff76e-a86f-4619-9455-9efd1b569f67 admin admin] The token cache is being invalidate because project cb01c37604464a0abefda50c0c9d7d51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:44:02.573069 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-9c7ff76e-a86f-4619-9455-9efd1b569f67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:02.574234 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-9c7ff76e-a86f-4619-9455-9efd1b569f67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:02.577115 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1180/2387] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:44:02 2022] DELETE /identity/v3/projects/cb01c37604464a0abefda50c0c9d7d51 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:44:02.582231 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf110bf-21b4-43c9-b9d3-73c08a82162c None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:02.582798 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf110bf-21b4-43c9-b9d3-73c08a82162c None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:02.583095 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf110bf-21b4-43c9-b9d3-73c08a82162c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:02.612328 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-2cf110bf-21b4-43c9-b9d3-73c08a82162c None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:44:02.614536 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-2cf110bf-21b4-43c9-b9d3-73c08a82162c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:02.708328 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1208/2388] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:44:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:44:02.717109 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b7bd4-cc76-4e67-9a4f-eae293b41084 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:02.718760 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b7bd4-cc76-4e67-9a4f-eae293b41084 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:02.719199 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b7bd4-cc76-4e67-9a4f-eae293b41084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:02.742654 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-f53b7bd4-cc76-4e67-9a4f-eae293b41084 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:44:02.744256 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-f53b7bd4-cc76-4e67-9a4f-eae293b41084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:02.754547 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1181/2389] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:44:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:44:02.759463 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf110bf-21b4-43c9-b9d3-73c08a82162c None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:02.771854 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf110bf-21b4-43c9-b9d3-73c08a82162c None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:02.788861 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43480bbc-a672-4509-a78f-e8d391848065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:02.788861 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43480bbc-a672-4509-a78f-e8d391848065 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:02.788861 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43480bbc-a672-4509-a78f-e8d391848065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:02.788861 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43480bbc-a672-4509-a78f-e8d391848065 admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:02.789413 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43480bbc-a672-4509-a78f-e8d391848065 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:44:02.789413 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43480bbc-a672-4509-a78f-e8d391848065 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:02.789413 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43480bbc-a672-4509-a78f-e8d391848065 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:02.812896 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1209/2390] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:44:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:44:02.826491 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53b7bd4-cc76-4e67-9a4f-eae293b41084 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:02.843787 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53b7bd4-cc76-4e67-9a4f-eae293b41084 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:02.856138 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b8619-f1c9-417e-a382-3b13bcec3373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:02.856854 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b8619-f1c9-417e-a382-3b13bcec3373 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:02.857222 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b8619-f1c9-417e-a382-3b13bcec3373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:02.857425 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b8619-f1c9-417e-a382-3b13bcec3373 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:02.858356 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407b8619-f1c9-417e-a382-3b13bcec3373 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:02.861819 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407b8619-f1c9-417e-a382-3b13bcec3373 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:02.918941 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1182/2391] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:44:02 2022] POST /identity/v3/projects => generated 373 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:44:02.924560 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43480bbc-a672-4509-a78f-e8d391848065 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:02.940645 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43480bbc-a672-4509-a78f-e8d391848065 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:03.056038 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de33405-9025-4bc3-a22c-71fafbfd577d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:03.056816 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de33405-9025-4bc3-a22c-71fafbfd577d admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:03.058181 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de33405-9025-4bc3-a22c-71fafbfd577d admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:03.058483 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de33405-9025-4bc3-a22c-71fafbfd577d admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:03.059204 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de33405-9025-4bc3-a22c-71fafbfd577d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:03.061196 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de33405-9025-4bc3-a22c-71fafbfd577d admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:03.098543 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1210/2392] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:44:02 2022] POST /identity/v3/users => generated 287 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:44:03.099897 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b8619-f1c9-417e-a382-3b13bcec3373 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:03.115138 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b8619-f1c9-417e-a382-3b13bcec3373 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:03.121893 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b396ebf1-bce6-44b2-992c-f7d08a2f86c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:03.123089 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b396ebf1-bce6-44b2-992c-f7d08a2f86c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:03.124141 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b396ebf1-bce6-44b2-992c-f7d08a2f86c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:03.124887 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b396ebf1-bce6-44b2-992c-f7d08a2f86c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:03.126022 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b396ebf1-bce6-44b2-992c-f7d08a2f86c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:03.133854 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b396ebf1-bce6-44b2-992c-f7d08a2f86c8 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:03.153482 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1183/2393] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:44:03 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:44:03.157994 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de33405-9025-4bc3-a22c-71fafbfd577d admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:03.186243 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de33405-9025-4bc3-a22c-71fafbfd577d admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:03.207087 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2b9478-f0f8-439d-ae08-88bfc3ad4aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:03.208107 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2b9478-f0f8-439d-ae08-88bfc3ad4aa4 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:03.208564 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2b9478-f0f8-439d-ae08-88bfc3ad4aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:03.209006 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2b9478-f0f8-439d-ae08-88bfc3ad4aa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:03.209985 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2b9478-f0f8-439d-ae08-88bfc3ad4aa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:03.211840 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2b9478-f0f8-439d-ae08-88bfc3ad4aa4 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:03.225968 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1211/2394] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:44:03 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:44:03.230140 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b396ebf1-bce6-44b2-992c-f7d08a2f86c8 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:03.244876 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b396ebf1-bce6-44b2-992c-f7d08a2f86c8 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:03.252069 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffaf86f-9f10-4590-b97e-851ed0501bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:03.253086 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffaf86f-9f10-4590-b97e-851ed0501bff admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:03.253471 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffaf86f-9f10-4590-b97e-851ed0501bff admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:03.253855 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffaf86f-9f10-4590-b97e-851ed0501bff admin admin] PATH_INFO: `/v3/projects/1950eb49f14e4983b0e64142d970d53e/users/0f332df6b4d743a3886e9519462225cc/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:03.281198 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffaf86f-9f10-4590-b97e-851ed0501bff admin admin] RBAC: Authorizing `identity:create_grant(project_id=1950eb49f14e4983b0e64142d970d53e, user_id=0f332df6b4d743a3886e9519462225cc, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:03.289091 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffaf86f-9f10-4590-b97e-851ed0501bff admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:03.319213 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1184/2395] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:44:03 2022] PUT /identity/v3/projects/1950eb49f14e4983b0e64142d970d53e/users/0f332df6b4d743a3886e9519462225cc/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:44:03.330417 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37450386-1c2e-418f-a054-bc43efb9cd1f None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:03.330791 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37450386-1c2e-418f-a054-bc43efb9cd1f None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:03.331078 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37450386-1c2e-418f-a054-bc43efb9cd1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:03.383079 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-37450386-1c2e-418f-a054-bc43efb9cd1f None None] MFA Rules not processed for user `0f332df6b4d743a3886e9519462225cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:44:03.383999 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-37450386-1c2e-418f-a054-bc43efb9cd1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:03.439467 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:44:03.493396 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1212/2396] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:44:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:44:03.501758 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffaf86f-9f10-4590-b97e-851ed0501bff admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:03.505251 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-dffaf86f-9f10-4590-b97e-851ed0501bff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:03.613944 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffaf86f-9f10-4590-b97e-851ed0501bff admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:03.617704 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f5dd78-ea66-4cf7-8461-9705a17de883 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:03.618373 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f5dd78-ea66-4cf7-8461-9705a17de883 service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:03.618661 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f5dd78-ea66-4cf7-8461-9705a17de883 service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:03.618941 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f5dd78-ea66-4cf7-8461-9705a17de883 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:03.633524 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f5dd78-ea66-4cf7-8461-9705a17de883 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:03.635054 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f5dd78-ea66-4cf7-8461-9705a17de883 service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:03.646847 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1185/2397] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:44:03 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:44:13.482487 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=9 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:44:13.641405 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=75,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:44:14.097139 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37450386-1c2e-418f-a054-bc43efb9cd1f None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:14.098666 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-37450386-1c2e-418f-a054-bc43efb9cd1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:14.117165 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f5dd78-ea66-4cf7-8461-9705a17de883 service cinder] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:14.172993 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37450386-1c2e-418f-a054-bc43efb9cd1f None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:14.177086 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc35097-4a7d-41b9-aa3d-c6fa414e6af6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:14.177916 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc35097-4a7d-41b9-aa3d-c6fa414e6af6 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:14.178272 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc35097-4a7d-41b9-aa3d-c6fa414e6af6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:14.179494 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc35097-4a7d-41b9-aa3d-c6fa414e6af6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:14.180971 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-ddc35097-4a7d-41b9-aa3d-c6fa414e6af6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:14.199133 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f5dd78-ea66-4cf7-8461-9705a17de883 service cinder] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:14.205821 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffb4015-ab91-441e-8070-df4c664c02dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:14.207057 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb4015-ab91-441e-8070-df4c664c02dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:14.207531 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb4015-ab91-441e-8070-df4c664c02dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:14.207974 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb4015-ab91-441e-8070-df4c664c02dd admin admin] PATH_INFO: `/v3/users/0f332df6b4d743a3886e9519462225cc` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:14.209363 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffb4015-ab91-441e-8070-df4c664c02dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f332df6b4d743a3886e9519462225cc)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:14.211514 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffb4015-ab91-441e-8070-df4c664c02dd admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:14.265482 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc35097-4a7d-41b9-aa3d-c6fa414e6af6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:14.266940 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc35097-4a7d-41b9-aa3d-c6fa414e6af6 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:14.270750 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-dffb4015-ab91-441e-8070-df4c664c02dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f332df6b4d743a3886e9519462225cc'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:14.277387 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1213/2398] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:44:14 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:44:14.288204 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-dffb4015-ab91-441e-8070-df4c664c02dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f332df6b4d743a3886e9519462225cc'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:14.289082 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-dffb4015-ab91-441e-8070-df4c664c02dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f332df6b4d743a3886e9519462225cc'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:14.296820 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-dffb4015-ab91-441e-8070-df4c664c02dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f332df6b4d743a3886e9519462225cc'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:14.297499 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-dffb4015-ab91-441e-8070-df4c664c02dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f332df6b4d743a3886e9519462225cc'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:14.312496 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1186/2399] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:44:14 2022] DELETE /identity/v3/users/0f332df6b4d743a3886e9519462225cc => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:44:14.317108 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b001c9ed-8d2a-409c-b683-f8adc1ed643e None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:14.317402 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b001c9ed-8d2a-409c-b683-f8adc1ed643e None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:14.317740 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b001c9ed-8d2a-409c-b683-f8adc1ed643e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:14.337263 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-b001c9ed-8d2a-409c-b683-f8adc1ed643e None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:44:14.338651 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b001c9ed-8d2a-409c-b683-f8adc1ed643e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:14.405878 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1214/2400] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:44:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:44:14.417516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffb4015-ab91-441e-8070-df4c664c02dd admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:14.420083 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-dffb4015-ab91-441e-8070-df4c664c02dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:14.496663 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffb4015-ab91-441e-8070-df4c664c02dd admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:14.501736 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20130b6-9864-4d31-bfcf-8c5c7226b22b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:14.502748 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20130b6-9864-4d31-bfcf-8c5c7226b22b service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:14.503157 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20130b6-9864-4d31-bfcf-8c5c7226b22b service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:14.503608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20130b6-9864-4d31-bfcf-8c5c7226b22b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:14.515962 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20130b6-9864-4d31-bfcf-8c5c7226b22b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:14.517334 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20130b6-9864-4d31-bfcf-8c5c7226b22b service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:14.528966 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1187/2401] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:44:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:44:14.896733 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b001c9ed-8d2a-409c-b683-f8adc1ed643e None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:14.900333 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b001c9ed-8d2a-409c-b683-f8adc1ed643e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:14.915680 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b001c9ed-8d2a-409c-b683-f8adc1ed643e None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:14.921383 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce8183-6084-4342-85c2-3dac0de07089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:14.921998 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce8183-6084-4342-85c2-3dac0de07089 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:14.922331 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce8183-6084-4342-85c2-3dac0de07089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:14.922606 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce8183-6084-4342-85c2-3dac0de07089 admin admin] PATH_INFO: `/v3/projects/1950eb49f14e4983b0e64142d970d53e` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:14.923836 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ce8183-6084-4342-85c2-3dac0de07089 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1950eb49f14e4983b0e64142d970d53e)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:14.925860 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ce8183-6084-4342-85c2-3dac0de07089 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:14.984503 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:44:14.989090 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-40ce8183-6084-4342-85c2-3dac0de07089 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1950eb49f14e4983b0e64142d970d53e', 'request_id': 'req-40ce8183-6084-4342-85c2-3dac0de07089'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:14.997168 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-40ce8183-6084-4342-85c2-3dac0de07089 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1950eb49f14e4983b0e64142d970d53e', 'request_id': 'req-40ce8183-6084-4342-85c2-3dac0de07089'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:15.004359 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-40ce8183-6084-4342-85c2-3dac0de07089 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1950eb49f14e4983b0e64142d970d53e', 'request_id': 'req-40ce8183-6084-4342-85c2-3dac0de07089'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:15.019590 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-40ce8183-6084-4342-85c2-3dac0de07089 admin admin] The token cache is being invalidate because project 1950eb49f14e4983b0e64142d970d53e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:44:15.019879 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-40ce8183-6084-4342-85c2-3dac0de07089 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:15.021058 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-40ce8183-6084-4342-85c2-3dac0de07089 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:15.022883 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1215/2402] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:44:14 2022] DELETE /identity/v3/projects/1950eb49f14e4983b0e64142d970d53e => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:44:21.015117 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af28bd82-89fb-4bbe-adb4-10d108184756 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:21.015681 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af28bd82-89fb-4bbe-adb4-10d108184756 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:21.015681 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af28bd82-89fb-4bbe-adb4-10d108184756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:21.033606 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-af28bd82-89fb-4bbe-adb4-10d108184756 None None] MFA Rules not processed for user `85d9c4c57681447c898ebfd7990b05b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:44:21.034877 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-af28bd82-89fb-4bbe-adb4-10d108184756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:21.054698 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:44:21.097002 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1188/2403] 10.222.0.13 () {58 vars in 967 bytes} [Fri Jun 24 08:44:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:44:21.110089 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce8183-6084-4342-85c2-3dac0de07089 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:21.111487 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-40ce8183-6084-4342-85c2-3dac0de07089 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:21.191638 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce8183-6084-4342-85c2-3dac0de07089 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:21.199223 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543e45f2-1365-4a18-b591-add5b9b759ad service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec8c2f113cf41a0939b81f38fbee813', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:21.199917 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e45f2-1365-4a18-b591-add5b9b759ad service swift] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:21.200206 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e45f2-1365-4a18-b591-add5b9b759ad service swift] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:21.200485 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e45f2-1365-4a18-b591-add5b9b759ad service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:21.210605 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543e45f2-1365-4a18-b591-add5b9b759ad service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:21.212237 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543e45f2-1365-4a18-b591-add5b9b759ad service swift] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:21.222608 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1216/2404] 10.222.0.13 () {60 vars in 1384 bytes} [Fri Jun 24 08:44:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 08:44:31.089751 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=15 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:44:31.217671 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:DELETE=4,keystone:SELECT=39,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:45:06.435349 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe82fec-3a03-4111-96fc-46d18a383dfe None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:45:06.436292 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe82fec-3a03-4111-96fc-46d18a383dfe None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:45:06.436292 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe82fec-3a03-4111-96fc-46d18a383dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:45:06.455759 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-6fe82fec-3a03-4111-96fc-46d18a383dfe None None] MFA Rules not processed for user `2039bddb840c4ac784f0b1c52326c79f`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:45:06.456989 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-6fe82fec-3a03-4111-96fc-46d18a383dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:45:06.516985 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1189/2405] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:45:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:45:16.509639 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=25 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:45:26.442881 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d6a696-21c7-4378-bf19-8a4ef94f2509 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:45:26.443466 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d6a696-21c7-4378-bf19-8a4ef94f2509 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:45:26.443736 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d6a696-21c7-4378-bf19-8a4ef94f2509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:45:26.464866 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-22d6a696-21c7-4378-bf19-8a4ef94f2509 None None] MFA Rules not processed for user `71cbad05d1a54b4b9959e2ebbcb720fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:45:26.466146 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-22d6a696-21c7-4378-bf19-8a4ef94f2509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:45:26.528749 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1217/2406] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:45:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:45:32.011644 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe82fec-3a03-4111-96fc-46d18a383dfe None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:45:32.012181 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d6a696-21c7-4378-bf19-8a4ef94f2509 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:45:32.013084 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-6fe82fec-3a03-4111-96fc-46d18a383dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:45:32.013215 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-22d6a696-21c7-4378-bf19-8a4ef94f2509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:45:32.075821 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe82fec-3a03-4111-96fc-46d18a383dfe None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:45:32.078304 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d6a696-21c7-4378-bf19-8a4ef94f2509 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:45:32.080443 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9825911b-8b4c-487b-8e94-fcc74dc184a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:45:32.081109 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9825911b-8b4c-487b-8e94-fcc74dc184a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:45:32.081395 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9825911b-8b4c-487b-8e94-fcc74dc184a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:45:32.081676 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9825911b-8b4c-487b-8e94-fcc74dc184a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:45:32.082211 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc4881a-65bf-4dc2-a72d-162f04d02e4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:45:32.082870 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc4881a-65bf-4dc2-a72d-162f04d02e4e service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:45:32.083153 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc4881a-65bf-4dc2-a72d-162f04d02e4e service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:45:32.083428 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc4881a-65bf-4dc2-a72d-162f04d02e4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:45:32.083527 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-9825911b-8b4c-487b-8e94-fcc74dc184a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:45:32.085212 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-8fc4881a-65bf-4dc2-a72d-162f04d02e4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:45:32.169498 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9825911b-8b4c-487b-8e94-fcc74dc184a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:45:32.171475 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9825911b-8b4c-487b-8e94-fcc74dc184a3 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:45:32.183779 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc4881a-65bf-4dc2-a72d-162f04d02e4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:45:32.186566 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc4881a-65bf-4dc2-a72d-162f04d02e4e service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:45:32.188089 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1190/2407] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:45:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:45:32.203733 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1218/2408] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:45:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:45:41.588541 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9825911b-8b4c-487b-8e94-fcc74dc184a3 service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:45:41.590859 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-9825911b-8b4c-487b-8e94-fcc74dc184a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:45:41.619352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9825911b-8b4c-487b-8e94-fcc74dc184a3 service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:45:41.629850 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdda1a5e-50b5-4ec2-9356-38a977df1bc7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74bb1bc6821b4500b26f160ccae5b7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:45:41.630795 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda1a5e-50b5-4ec2-9356-38a977df1bc7 service nova] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:45:41.631147 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda1a5e-50b5-4ec2-9356-38a977df1bc7 service nova] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:45:41.631488 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda1a5e-50b5-4ec2-9356-38a977df1bc7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:45:41.633609 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-bdda1a5e-50b5-4ec2-9356-38a977df1bc7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:45:41.742209 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdda1a5e-50b5-4ec2-9356-38a977df1bc7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:45:41.743864 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdda1a5e-50b5-4ec2-9356-38a977df1bc7 service nova] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:45:41.757851 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1191/2409] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:45:41 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:45:42.196275 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=79 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:45:45.434525 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5770a0a2-dbd8-4013-985b-a965439ea52c None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:45:45.435069 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5770a0a2-dbd8-4013-985b-a965439ea52c None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:45:45.435390 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5770a0a2-dbd8-4013-985b-a965439ea52c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:45:45.467585 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-5770a0a2-dbd8-4013-985b-a965439ea52c None None] MFA Rules not processed for user `75a612089b2b46a6b3ff5a6cd9e59579`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:45:45.469860 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-5770a0a2-dbd8-4013-985b-a965439ea52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:45:45.481890 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1219/2410] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:45:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:45:48.253879 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269b3d3-3249-49ec-8fc5-3813291606c5 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:45:48.254657 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269b3d3-3249-49ec-8fc5-3813291606c5 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:45:48.254657 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269b3d3-3249-49ec-8fc5-3813291606c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:45:48.275535 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-0269b3d3-3249-49ec-8fc5-3813291606c5 None None] MFA Rules not processed for user `71e0fd73e5c9430c813af1de9ccd4a52`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:45:48.277186 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-0269b3d3-3249-49ec-8fc5-3813291606c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:45:48.326072 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:45:48.376085 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1192/2411] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:45:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:45:55.452560 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:45:58.358964 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=7 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:46:06.895852 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5770a0a2-dbd8-4013-985b-a965439ea52c None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:46:06.897338 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-5770a0a2-dbd8-4013-985b-a965439ea52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:46:06.910296 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5770a0a2-dbd8-4013-985b-a965439ea52c None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:46:06.914067 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7b69b3-e101-475d-9be3-e0d7f5d50bc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:46:06.914891 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b69b3-e101-475d-9be3-e0d7f5d50bc1 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:46:06.915246 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b69b3-e101-475d-9be3-e0d7f5d50bc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:46:06.915594 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b69b3-e101-475d-9be3-e0d7f5d50bc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:46:06.917481 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-cb7b69b3-e101-475d-9be3-e0d7f5d50bc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:46:06.928022 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7b69b3-e101-475d-9be3-e0d7f5d50bc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:46:06.929467 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7b69b3-e101-475d-9be3-e0d7f5d50bc1 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:46:06.938066 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1220/2412] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:46:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:46:13.614661 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0269b3d3-3249-49ec-8fc5-3813291606c5 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:46:13.616146 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-0269b3d3-3249-49ec-8fc5-3813291606c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:46:13.685807 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7b69b3-e101-475d-9be3-e0d7f5d50bc1 service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:46:13.688668 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-cb7b69b3-e101-475d-9be3-e0d7f5d50bc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:46:13.704699 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0269b3d3-3249-49ec-8fc5-3813291606c5 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:46:13.712722 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13634520-697c-4464-90c8-3b51fb816066 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39cc268bad604dd08594e3b39c7ecf5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:46:13.714801 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13634520-697c-4464-90c8-3b51fb816066 service placement] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:46:13.716654 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13634520-697c-4464-90c8-3b51fb816066 service placement] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:46:13.717378 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13634520-697c-4464-90c8-3b51fb816066 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:46:13.720501 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-13634520-697c-4464-90c8-3b51fb816066 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:46:13.733774 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13634520-697c-4464-90c8-3b51fb816066 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:46:13.735576 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13634520-697c-4464-90c8-3b51fb816066 service placement] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:46:13.746120 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1193/2413] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:46:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:46:13.761346 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7b69b3-e101-475d-9be3-e0d7f5d50bc1 service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:46:13.766052 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c628301-a343-44d0-892d-a97ebdc6fd54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:46:13.766920 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c628301-a343-44d0-892d-a97ebdc6fd54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:46:13.767206 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c628301-a343-44d0-892d-a97ebdc6fd54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:46:13.767485 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c628301-a343-44d0-892d-a97ebdc6fd54 admin admin] PATH_INFO: `/v3/users/2039bddb840c4ac784f0b1c52326c79f` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:46:13.768716 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c628301-a343-44d0-892d-a97ebdc6fd54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2039bddb840c4ac784f0b1c52326c79f)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:46:13.770665 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c628301-a343-44d0-892d-a97ebdc6fd54 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:46:13.823527 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7c628301-a343-44d0-892d-a97ebdc6fd54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2039bddb840c4ac784f0b1c52326c79f'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:46:13.835864 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7c628301-a343-44d0-892d-a97ebdc6fd54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2039bddb840c4ac784f0b1c52326c79f'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:46:13.843810 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7c628301-a343-44d0-892d-a97ebdc6fd54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2039bddb840c4ac784f0b1c52326c79f'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:46:13.844603 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7c628301-a343-44d0-892d-a97ebdc6fd54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2039bddb840c4ac784f0b1c52326c79f'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:46:13.845215 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7c628301-a343-44d0-892d-a97ebdc6fd54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2039bddb840c4ac784f0b1c52326c79f'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:46:13.857641 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1221/2414] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:46:13 2022] DELETE /identity/v3/users/2039bddb840c4ac784f0b1c52326c79f => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:46:14.034802 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13634520-697c-4464-90c8-3b51fb816066 service placement] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:46:14.047511 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13634520-697c-4464-90c8-3b51fb816066 service placement] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:46:14.107146 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b195877e-5936-4be0-a92f-eff7e7f3e541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:46:14.108618 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195877e-5936-4be0-a92f-eff7e7f3e541 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:46:14.108899 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195877e-5936-4be0-a92f-eff7e7f3e541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:46:14.109180 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195877e-5936-4be0-a92f-eff7e7f3e541 admin admin] PATH_INFO: `/v3/projects/84c244fc43f94ea2bc9f76965c111e78` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:46:14.110453 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b195877e-5936-4be0-a92f-eff7e7f3e541 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84c244fc43f94ea2bc9f76965c111e78)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:46:14.112546 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b195877e-5936-4be0-a92f-eff7e7f3e541 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:46:14.184685 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-b195877e-5936-4be0-a92f-eff7e7f3e541 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84c244fc43f94ea2bc9f76965c111e78', 'request_id': 'req-b195877e-5936-4be0-a92f-eff7e7f3e541'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:46:14.191317 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-b195877e-5936-4be0-a92f-eff7e7f3e541 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84c244fc43f94ea2bc9f76965c111e78', 'request_id': 'req-b195877e-5936-4be0-a92f-eff7e7f3e541'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:46:14.198491 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-b195877e-5936-4be0-a92f-eff7e7f3e541 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84c244fc43f94ea2bc9f76965c111e78', 'request_id': 'req-b195877e-5936-4be0-a92f-eff7e7f3e541'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:46:14.219805 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-b195877e-5936-4be0-a92f-eff7e7f3e541 admin admin] The token cache is being invalidate because project 84c244fc43f94ea2bc9f76965c111e78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:46:14.220048 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-b195877e-5936-4be0-a92f-eff7e7f3e541 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:46:14.220831 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-b195877e-5936-4be0-a92f-eff7e7f3e541 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:46:14.222483 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1194/2415] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:46:14 2022] DELETE /identity/v3/projects/84c244fc43f94ea2bc9f76965c111e78 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:46:22.952131 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c628301-a343-44d0-892d-a97ebdc6fd54 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:46:22.954189 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-7c628301-a343-44d0-892d-a97ebdc6fd54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:46:23.038355 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:46:23.044530 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c628301-a343-44d0-892d-a97ebdc6fd54 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:46:23.120181 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cdad53-3bde-4f1f-8610-e90f4bd312cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74bb1bc6821b4500b26f160ccae5b7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:46:23.120852 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cdad53-3bde-4f1f-8610-e90f4bd312cb service nova] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:46:23.121135 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cdad53-3bde-4f1f-8610-e90f4bd312cb service nova] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:46:23.121414 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cdad53-3bde-4f1f-8610-e90f4bd312cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:46:23.124147 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-13cdad53-3bde-4f1f-8610-e90f4bd312cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:46:23.186896 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cdad53-3bde-4f1f-8610-e90f4bd312cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:46:23.188508 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cdad53-3bde-4f1f-8610-e90f4bd312cb service nova] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:46:23.199534 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1222/2416] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:46:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:46:24.215656 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:46:31.816842 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b195877e-5936-4be0-a92f-eff7e7f3e541 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:46:31.818974 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-b195877e-5936-4be0-a92f-eff7e7f3e541 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:46:31.883125 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b195877e-5936-4be0-a92f-eff7e7f3e541 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:46:31.886777 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45468783-32fd-4aa1-b405-3f7dda4c3a34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:46:31.887422 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45468783-32fd-4aa1-b405-3f7dda4c3a34 service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:46:31.887711 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45468783-32fd-4aa1-b405-3f7dda4c3a34 service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:46:31.887986 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45468783-32fd-4aa1-b405-3f7dda4c3a34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:46:31.890444 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-45468783-32fd-4aa1-b405-3f7dda4c3a34 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:46:31.950763 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45468783-32fd-4aa1-b405-3f7dda4c3a34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:46:31.952370 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45468783-32fd-4aa1-b405-3f7dda4c3a34 service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:46:31.961415 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1195/2417] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:46:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:46:33.193712 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=54 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:46:41.355042 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cdad53-3bde-4f1f-8610-e90f4bd312cb service nova] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:46:41.361370 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-13cdad53-3bde-4f1f-8610-e90f4bd312cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:46:41.404032 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cdad53-3bde-4f1f-8610-e90f4bd312cb service nova] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:46:41.412076 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f661e86-8ad4-483b-a64f-c4a21eeddfe9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74bb1bc6821b4500b26f160ccae5b7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:46:41.412786 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f661e86-8ad4-483b-a64f-c4a21eeddfe9 service nova] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:46:41.413263 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f661e86-8ad4-483b-a64f-c4a21eeddfe9 service nova] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:46:41.413591 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f661e86-8ad4-483b-a64f-c4a21eeddfe9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:46:41.415743 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-4f661e86-8ad4-483b-a64f-c4a21eeddfe9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:46:41.437364 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f661e86-8ad4-483b-a64f-c4a21eeddfe9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:46:41.439550 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f661e86-8ad4-483b-a64f-c4a21eeddfe9 service nova] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:46:41.468877 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1223/2418] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:46:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:46:41.957465 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=48 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:46:42.188308 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45468783-32fd-4aa1-b405-3f7dda4c3a34 service cinder] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:46:42.203450 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45468783-32fd-4aa1-b405-3f7dda4c3a34 service cinder] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:46:42.209934 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6079a94f-adae-4115-b1d0-f042f8668450 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74bb1bc6821b4500b26f160ccae5b7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:46:42.211010 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6079a94f-adae-4115-b1d0-f042f8668450 service nova] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:46:42.211460 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6079a94f-adae-4115-b1d0-f042f8668450 service nova] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:46:42.211898 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6079a94f-adae-4115-b1d0-f042f8668450 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:46:42.221899 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-6079a94f-adae-4115-b1d0-f042f8668450 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:46:42.269348 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6079a94f-adae-4115-b1d0-f042f8668450 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:46:42.277035 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6079a94f-adae-4115-b1d0-f042f8668450 service nova] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:46:42.302920 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1196/2419] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:46:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:46:43.749612 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23fedee-21a3-44b0-bc48-e03e26557605 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:46:43.750358 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23fedee-21a3-44b0-bc48-e03e26557605 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:46:43.751475 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23fedee-21a3-44b0-bc48-e03e26557605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:46:43.803512 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-b23fedee-21a3-44b0-bc48-e03e26557605 None None] MFA Rules not processed for user `bf905723382e4372ae63d77a376982ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:46:43.806364 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b23fedee-21a3-44b0-bc48-e03e26557605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:46:43.899316 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1224/2420] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:46:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:46:44.319534 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6079a94f-adae-4115-b1d0-f042f8668450 service nova] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:46:44.321232 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-6079a94f-adae-4115-b1d0-f042f8668450 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:46:44.417490 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6079a94f-adae-4115-b1d0-f042f8668450 service nova] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:46:44.425667 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c374fc53-ae37-47ca-b56b-1449b982161e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:46:44.426739 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c374fc53-ae37-47ca-b56b-1449b982161e service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:46:44.427065 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c374fc53-ae37-47ca-b56b-1449b982161e service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:46:44.427347 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c374fc53-ae37-47ca-b56b-1449b982161e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:46:44.429412 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-c374fc53-ae37-47ca-b56b-1449b982161e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:46:44.443411 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c374fc53-ae37-47ca-b56b-1449b982161e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:46:44.444832 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c374fc53-ae37-47ca-b56b-1449b982161e service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:46:44.456667 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1197/2421] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:46:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:46:45.057894 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23fedee-21a3-44b0-bc48-e03e26557605 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:46:45.061678 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b23fedee-21a3-44b0-bc48-e03e26557605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:46:45.169240 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23fedee-21a3-44b0-bc48-e03e26557605 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:46:45.175691 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4648c65c-6000-4d52-8ab6-b062ac1e6099 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39cc268bad604dd08594e3b39c7ecf5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:46:45.176435 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648c65c-6000-4d52-8ab6-b062ac1e6099 service placement] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:46:45.176769 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648c65c-6000-4d52-8ab6-b062ac1e6099 service placement] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:46:45.177095 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648c65c-6000-4d52-8ab6-b062ac1e6099 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:46:45.179352 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-4648c65c-6000-4d52-8ab6-b062ac1e6099 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:46:45.193079 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4648c65c-6000-4d52-8ab6-b062ac1e6099 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:46:45.194815 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4648c65c-6000-4d52-8ab6-b062ac1e6099 service placement] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:46:45.207335 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1225/2422] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:46:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:46:54.452343 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=32 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:46:55.202088 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=63 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:46:57.105299 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c374fc53-ae37-47ca-b56b-1449b982161e service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:46:57.106871 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-c374fc53-ae37-47ca-b56b-1449b982161e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:46:57.118800 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c374fc53-ae37-47ca-b56b-1449b982161e service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:46:57.123223 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eed8b9-c515-4ffb-8b18-6f1da4a62163 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:46:57.123957 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eed8b9-c515-4ffb-8b18-6f1da4a62163 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:46:57.124305 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eed8b9-c515-4ffb-8b18-6f1da4a62163 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:46:57.124644 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eed8b9-c515-4ffb-8b18-6f1da4a62163 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:46:57.126865 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-50eed8b9-c515-4ffb-8b18-6f1da4a62163 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:46:57.210241 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eed8b9-c515-4ffb-8b18-6f1da4a62163 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:46:57.212333 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eed8b9-c515-4ffb-8b18-6f1da4a62163 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:46:57.224542 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1198/2423] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:46:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:46:59.726193 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4648c65c-6000-4d52-8ab6-b062ac1e6099 service placement] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:46:59.744112 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4648c65c-6000-4d52-8ab6-b062ac1e6099 service placement] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:46:59.750842 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eb94ce-d36f-4923-b335-42c5974f4457 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:46:59.752227 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb94ce-d36f-4923-b335-42c5974f4457 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:46:59.752750 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb94ce-d36f-4923-b335-42c5974f4457 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:46:59.753261 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb94ce-d36f-4923-b335-42c5974f4457 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:46:59.756376 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-a7eb94ce-d36f-4923-b335-42c5974f4457 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:46:59.781752 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eb94ce-d36f-4923-b335-42c5974f4457 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:46:59.785087 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eb94ce-d36f-4923-b335-42c5974f4457 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:46:59.817766 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1226/2424] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:46:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:47:01.881000 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eed8b9-c515-4ffb-8b18-6f1da4a62163 service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:01.882429 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-50eed8b9-c515-4ffb-8b18-6f1da4a62163 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:01.905202 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eed8b9-c515-4ffb-8b18-6f1da4a62163 service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:01.912012 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa50663-8bb9-40e8-837c-dc421471c420 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39cc268bad604dd08594e3b39c7ecf5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:01.912688 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa50663-8bb9-40e8-837c-dc421471c420 service placement] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:01.912976 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa50663-8bb9-40e8-837c-dc421471c420 service placement] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:01.913265 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa50663-8bb9-40e8-837c-dc421471c420 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:01.915308 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-dfa50663-8bb9-40e8-837c-dc421471c420 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:01.930870 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa50663-8bb9-40e8-837c-dc421471c420 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:01.932524 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa50663-8bb9-40e8-837c-dc421471c420 service placement] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:01.958465 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1199/2425] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:47:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:47:02.054424 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eb94ce-d36f-4923-b335-42c5974f4457 service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:02.058880 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-a7eb94ce-d36f-4923-b335-42c5974f4457 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:02.098006 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eb94ce-d36f-4923-b335-42c5974f4457 service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:02.116367 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9994c4a6-efb9-47f7-a801-ceeb2bcb6034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:02.118097 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994c4a6-efb9-47f7-a801-ceeb2bcb6034 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:02.119686 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994c4a6-efb9-47f7-a801-ceeb2bcb6034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:02.120354 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994c4a6-efb9-47f7-a801-ceeb2bcb6034 admin admin] PATH_INFO: `/v3/users/71e0fd73e5c9430c813af1de9ccd4a52` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:02.124539 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9994c4a6-efb9-47f7-a801-ceeb2bcb6034 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71e0fd73e5c9430c813af1de9ccd4a52)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:02.130088 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9994c4a6-efb9-47f7-a801-ceeb2bcb6034 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:02.306852 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-9994c4a6-efb9-47f7-a801-ceeb2bcb6034 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71e0fd73e5c9430c813af1de9ccd4a52'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:02.355176 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-9994c4a6-efb9-47f7-a801-ceeb2bcb6034 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71e0fd73e5c9430c813af1de9ccd4a52'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:02.382647 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-9994c4a6-efb9-47f7-a801-ceeb2bcb6034 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71e0fd73e5c9430c813af1de9ccd4a52'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:02.384224 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-9994c4a6-efb9-47f7-a801-ceeb2bcb6034 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71e0fd73e5c9430c813af1de9ccd4a52'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:02.385579 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-9994c4a6-efb9-47f7-a801-ceeb2bcb6034 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71e0fd73e5c9430c813af1de9ccd4a52'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:02.419002 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1227/2426] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:47:02 2022] DELETE /identity/v3/users/71e0fd73e5c9430c813af1de9ccd4a52 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:47:02.696510 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa50663-8bb9-40e8-837c-dc421471c420 service placement] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:02.711393 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa50663-8bb9-40e8-837c-dc421471c420 service placement] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:02.870745 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1497a81-c594-488a-aa25-3ad20893d08f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:02.873895 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1497a81-c594-488a-aa25-3ad20893d08f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:02.874455 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1497a81-c594-488a-aa25-3ad20893d08f admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:02.875051 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1497a81-c594-488a-aa25-3ad20893d08f admin admin] PATH_INFO: `/v3/projects/dfa535df5dd14d69bb3ca125cbd9fe13` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:02.876868 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1497a81-c594-488a-aa25-3ad20893d08f admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfa535df5dd14d69bb3ca125cbd9fe13)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:02.880342 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1497a81-c594-488a-aa25-3ad20893d08f admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:02.974624 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f1497a81-c594-488a-aa25-3ad20893d08f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfa535df5dd14d69bb3ca125cbd9fe13', 'request_id': 'req-f1497a81-c594-488a-aa25-3ad20893d08f'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:02.986422 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f1497a81-c594-488a-aa25-3ad20893d08f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfa535df5dd14d69bb3ca125cbd9fe13', 'request_id': 'req-f1497a81-c594-488a-aa25-3ad20893d08f'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:02.996448 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f1497a81-c594-488a-aa25-3ad20893d08f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfa535df5dd14d69bb3ca125cbd9fe13', 'request_id': 'req-f1497a81-c594-488a-aa25-3ad20893d08f'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:03.020188 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f1497a81-c594-488a-aa25-3ad20893d08f admin admin] The token cache is being invalidate because project dfa535df5dd14d69bb3ca125cbd9fe13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:47:03.021440 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f1497a81-c594-488a-aa25-3ad20893d08f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:03.022483 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f1497a81-c594-488a-aa25-3ad20893d08f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:03.025807 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1200/2427] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:47:02 2022] DELETE /identity/v3/projects/dfa535df5dd14d69bb3ca125cbd9fe13 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:47:03.033552 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf83b20a-e0f5-4c12-a4ef-3939fccc1824 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:03.034085 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf83b20a-e0f5-4c12-a4ef-3939fccc1824 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:03.034658 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf83b20a-e0f5-4c12-a4ef-3939fccc1824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:03.066314 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-bf83b20a-e0f5-4c12-a4ef-3939fccc1824 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:47:03.068321 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-bf83b20a-e0f5-4c12-a4ef-3939fccc1824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:03.209197 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1228/2428] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:47:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:47:03.217204 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2e0059-05b4-497c-8def-e3ec30c10976 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:03.217748 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2e0059-05b4-497c-8def-e3ec30c10976 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:03.218116 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2e0059-05b4-497c-8def-e3ec30c10976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:03.242961 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-6c2e0059-05b4-497c-8def-e3ec30c10976 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:47:03.244767 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-6c2e0059-05b4-497c-8def-e3ec30c10976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:03.255302 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1201/2429] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:47:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:47:03.260641 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf83b20a-e0f5-4c12-a4ef-3939fccc1824 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:03.279181 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf83b20a-e0f5-4c12-a4ef-3939fccc1824 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:03.287163 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f62a78-ca0a-4390-b816-128b5bfdc87f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:03.288236 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f62a78-ca0a-4390-b816-128b5bfdc87f admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:03.288714 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f62a78-ca0a-4390-b816-128b5bfdc87f admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:03.289190 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f62a78-ca0a-4390-b816-128b5bfdc87f admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:03.290543 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f62a78-ca0a-4390-b816-128b5bfdc87f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:47:03.291069 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f62a78-ca0a-4390-b816-128b5bfdc87f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:03.294164 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f62a78-ca0a-4390-b816-128b5bfdc87f admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:03.320486 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1229/2430] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:47:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:47:03.322000 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2e0059-05b4-497c-8def-e3ec30c10976 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:03.330950 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:47:03.338495 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2e0059-05b4-497c-8def-e3ec30c10976 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:03.350170 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3107eb-c599-4abd-a9e7-c1399465ae52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:03.351701 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3107eb-c599-4abd-a9e7-c1399465ae52 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:03.352225 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3107eb-c599-4abd-a9e7-c1399465ae52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:03.352727 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3107eb-c599-4abd-a9e7-c1399465ae52 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:03.354071 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3107eb-c599-4abd-a9e7-c1399465ae52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:03.358481 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3107eb-c599-4abd-a9e7-c1399465ae52 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:03.441815 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1202/2431] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:47:03 2022] POST /identity/v3/projects => generated 383 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:47:03.443896 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f62a78-ca0a-4390-b816-128b5bfdc87f admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:03.463856 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f62a78-ca0a-4390-b816-128b5bfdc87f admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:03.545409 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:47:03.571428 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43047960-3381-46b9-9cd2-19bf2782ef96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:03.572147 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43047960-3381-46b9-9cd2-19bf2782ef96 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:03.572434 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43047960-3381-46b9-9cd2-19bf2782ef96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:03.572711 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43047960-3381-46b9-9cd2-19bf2782ef96 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:03.574222 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43047960-3381-46b9-9cd2-19bf2782ef96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:03.575633 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43047960-3381-46b9-9cd2-19bf2782ef96 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:03.600684 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1230/2432] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:47:03 2022] POST /identity/v3/users => generated 292 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:47:03.606915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3107eb-c599-4abd-a9e7-c1399465ae52 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:03.620993 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3107eb-c599-4abd-a9e7-c1399465ae52 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:03.628572 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d90fc0f-d728-4446-b9ce-3972f18b7f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:03.629635 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90fc0f-d728-4446-b9ce-3972f18b7f60 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:03.630097 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90fc0f-d728-4446-b9ce-3972f18b7f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:03.630559 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90fc0f-d728-4446-b9ce-3972f18b7f60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:03.631458 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d90fc0f-d728-4446-b9ce-3972f18b7f60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:03.634490 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d90fc0f-d728-4446-b9ce-3972f18b7f60 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:03.652466 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1203/2433] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:47:03 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:47:03.659578 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43047960-3381-46b9-9cd2-19bf2782ef96 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:03.675540 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43047960-3381-46b9-9cd2-19bf2782ef96 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:03.681502 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c4eb0-4337-4bcf-8e57-6bf95c46b850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:03.682710 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c4eb0-4337-4bcf-8e57-6bf95c46b850 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:03.683281 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c4eb0-4337-4bcf-8e57-6bf95c46b850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:03.684248 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c4eb0-4337-4bcf-8e57-6bf95c46b850 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:03.687827 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6c4eb0-4337-4bcf-8e57-6bf95c46b850 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:03.690931 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6c4eb0-4337-4bcf-8e57-6bf95c46b850 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:03.717302 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1231/2434] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:47:03 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:47:03.719547 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d90fc0f-d728-4446-b9ce-3972f18b7f60 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:03.733852 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d90fc0f-d728-4446-b9ce-3972f18b7f60 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:03.740233 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffe1a2f-24bd-48d6-afad-22bc7d9458b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:03.741587 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe1a2f-24bd-48d6-afad-22bc7d9458b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:03.742079 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe1a2f-24bd-48d6-afad-22bc7d9458b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:03.742771 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe1a2f-24bd-48d6-afad-22bc7d9458b2 admin admin] PATH_INFO: `/v3/projects/7787ee132e9e4e23b1407718fb366406/users/8d15a76041274e7381087bd21eb674c9/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:03.765739 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffe1a2f-24bd-48d6-afad-22bc7d9458b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7787ee132e9e4e23b1407718fb366406, user_id=8d15a76041274e7381087bd21eb674c9, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:03.769774 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffe1a2f-24bd-48d6-afad-22bc7d9458b2 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:03.792834 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1204/2435] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:47:03 2022] PUT /identity/v3/projects/7787ee132e9e4e23b1407718fb366406/users/8d15a76041274e7381087bd21eb674c9/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:47:03.800702 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f2ed36-bf7f-4fa4-8386-f5ee3120283a None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:03.801281 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f2ed36-bf7f-4fa4-8386-f5ee3120283a None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:03.802105 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f2ed36-bf7f-4fa4-8386-f5ee3120283a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:03.853260 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-61f2ed36-bf7f-4fa4-8386-f5ee3120283a None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:47:03.856894 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-61f2ed36-bf7f-4fa4-8386-f5ee3120283a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:03.997303 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1232/2436] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:47:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:47:04.014697 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffe1a2f-24bd-48d6-afad-22bc7d9458b2 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:04.018476 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-3ffe1a2f-24bd-48d6-afad-22bc7d9458b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:04.162709 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffe1a2f-24bd-48d6-afad-22bc7d9458b2 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:04.174208 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b1f94e-2a2d-49d6-89c2-64b189f800cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:04.175641 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b1f94e-2a2d-49d6-89c2-64b189f800cd service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:04.176495 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b1f94e-2a2d-49d6-89c2-64b189f800cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:04.176991 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b1f94e-2a2d-49d6-89c2-64b189f800cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:04.194551 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b1f94e-2a2d-49d6-89c2-64b189f800cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:04.196632 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b1f94e-2a2d-49d6-89c2-64b189f800cd service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:04.225226 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1205/2437] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:47:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:47:04.364711 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f2ed36-bf7f-4fa4-8386-f5ee3120283a None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:04.377677 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-61f2ed36-bf7f-4fa4-8386-f5ee3120283a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:04.584070 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f2ed36-bf7f-4fa4-8386-f5ee3120283a None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:04.603923 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d50eb8-9f9d-450a-aca0-6bd7ff993168 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74bb1bc6821b4500b26f160ccae5b7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:04.605991 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d50eb8-9f9d-450a-aca0-6bd7ff993168 service nova] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:04.607715 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d50eb8-9f9d-450a-aca0-6bd7ff993168 service nova] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:04.608170 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d50eb8-9f9d-450a-aca0-6bd7ff993168 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:04.618374 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b9d50eb8-9f9d-450a-aca0-6bd7ff993168 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:04.661428 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d50eb8-9f9d-450a-aca0-6bd7ff993168 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:04.663780 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d50eb8-9f9d-450a-aca0-6bd7ff993168 service nova] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:04.692373 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1233/2438] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:47:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:47:07.681252 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b1f94e-2a2d-49d6-89c2-64b189f800cd service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:07.685998 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-a1b1f94e-2a2d-49d6-89c2-64b189f800cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:07.715057 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b1f94e-2a2d-49d6-89c2-64b189f800cd service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:07.724041 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e389efb7-f142-4b15-b77c-e37a21b7dc02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:07.725863 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e389efb7-f142-4b15-b77c-e37a21b7dc02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:07.726222 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e389efb7-f142-4b15-b77c-e37a21b7dc02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:07.726601 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e389efb7-f142-4b15-b77c-e37a21b7dc02 admin admin] PATH_INFO: `/v3/users/71cbad05d1a54b4b9959e2ebbcb720fd` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:07.728871 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e389efb7-f142-4b15-b77c-e37a21b7dc02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71cbad05d1a54b4b9959e2ebbcb720fd)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:07.732590 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e389efb7-f142-4b15-b77c-e37a21b7dc02 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:07.821432 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-e389efb7-f142-4b15-b77c-e37a21b7dc02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71cbad05d1a54b4b9959e2ebbcb720fd'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:07.844227 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-e389efb7-f142-4b15-b77c-e37a21b7dc02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71cbad05d1a54b4b9959e2ebbcb720fd'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:07.845073 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-e389efb7-f142-4b15-b77c-e37a21b7dc02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71cbad05d1a54b4b9959e2ebbcb720fd'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:07.855272 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-e389efb7-f142-4b15-b77c-e37a21b7dc02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71cbad05d1a54b4b9959e2ebbcb720fd'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:07.856086 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-e389efb7-f142-4b15-b77c-e37a21b7dc02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71cbad05d1a54b4b9959e2ebbcb720fd'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:07.882586 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1206/2439] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:47:07 2022] DELETE /identity/v3/users/71cbad05d1a54b4b9959e2ebbcb720fd => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:47:08.242459 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d50eb8-9f9d-450a-aca0-6bd7ff993168 service nova] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:08.270906 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d50eb8-9f9d-450a-aca0-6bd7ff993168 service nova] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:08.428770 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:47:08.452671 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a9ad04-ac93-4707-a996-5708096dd2f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:08.454133 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a9ad04-ac93-4707-a996-5708096dd2f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:08.454579 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a9ad04-ac93-4707-a996-5708096dd2f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:08.455009 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a9ad04-ac93-4707-a996-5708096dd2f9 admin admin] PATH_INFO: `/v3/projects/2988ac4c59124e02bbcc7049e69dd599` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:08.457510 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a9ad04-ac93-4707-a996-5708096dd2f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2988ac4c59124e02bbcc7049e69dd599)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:08.460994 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a9ad04-ac93-4707-a996-5708096dd2f9 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:08.531508 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-84a9ad04-ac93-4707-a996-5708096dd2f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2988ac4c59124e02bbcc7049e69dd599', 'request_id': 'req-84a9ad04-ac93-4707-a996-5708096dd2f9'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:08.541166 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-84a9ad04-ac93-4707-a996-5708096dd2f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2988ac4c59124e02bbcc7049e69dd599', 'request_id': 'req-84a9ad04-ac93-4707-a996-5708096dd2f9'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:08.552760 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-84a9ad04-ac93-4707-a996-5708096dd2f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2988ac4c59124e02bbcc7049e69dd599', 'request_id': 'req-84a9ad04-ac93-4707-a996-5708096dd2f9'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:08.597498 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-84a9ad04-ac93-4707-a996-5708096dd2f9 admin admin] The token cache is being invalidate because project 2988ac4c59124e02bbcc7049e69dd599 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:47:08.597896 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-84a9ad04-ac93-4707-a996-5708096dd2f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:08.599578 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-84a9ad04-ac93-4707-a996-5708096dd2f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:08.603619 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1234/2440] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:47:08 2022] DELETE /identity/v3/projects/2988ac4c59124e02bbcc7049e69dd599 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:47:08.611716 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e9027e-9e31-4dba-94ba-a2e45038bcb0 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:08.612017 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e9027e-9e31-4dba-94ba-a2e45038bcb0 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:08.612491 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e9027e-9e31-4dba-94ba-a2e45038bcb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:08.668260 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-89e9027e-9e31-4dba-94ba-a2e45038bcb0 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:47:08.673527 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-89e9027e-9e31-4dba-94ba-a2e45038bcb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:08.691863 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:47:08.802417 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1207/2441] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:47:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:47:08.813387 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74632d6-9a2b-4599-bf09-0c6d59ed8eb7 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:08.813387 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74632d6-9a2b-4599-bf09-0c6d59ed8eb7 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:08.813387 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74632d6-9a2b-4599-bf09-0c6d59ed8eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:08.838632 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-d74632d6-9a2b-4599-bf09-0c6d59ed8eb7 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:47:08.839871 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-d74632d6-9a2b-4599-bf09-0c6d59ed8eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:08.858233 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1235/2442] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:47:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:47:08.860314 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e9027e-9e31-4dba-94ba-a2e45038bcb0 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:08.880436 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e9027e-9e31-4dba-94ba-a2e45038bcb0 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:08.890201 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fcea0d-e47b-4aef-8e5d-a6849a2034a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:08.891580 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fcea0d-e47b-4aef-8e5d-a6849a2034a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:08.892334 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fcea0d-e47b-4aef-8e5d-a6849a2034a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:08.893045 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fcea0d-e47b-4aef-8e5d-a6849a2034a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:08.894874 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fcea0d-e47b-4aef-8e5d-a6849a2034a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:47:08.895646 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fcea0d-e47b-4aef-8e5d-a6849a2034a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:08.899029 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fcea0d-e47b-4aef-8e5d-a6849a2034a4 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:08.931238 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1208/2443] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:47:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:47:08.932183 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74632d6-9a2b-4599-bf09-0c6d59ed8eb7 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:08.944656 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74632d6-9a2b-4599-bf09-0c6d59ed8eb7 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:08.952464 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31242a4-2c54-457d-84f0-d2a388c006e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:08.953538 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31242a4-2c54-457d-84f0-d2a388c006e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:08.954022 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31242a4-2c54-457d-84f0-d2a388c006e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:08.954432 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31242a4-2c54-457d-84f0-d2a388c006e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:08.955418 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31242a4-2c54-457d-84f0-d2a388c006e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:08.957951 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31242a4-2c54-457d-84f0-d2a388c006e8 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:09.000346 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1236/2444] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:47:08 2022] POST /identity/v3/projects => generated 377 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:47:09.002560 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fcea0d-e47b-4aef-8e5d-a6849a2034a4 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:09.014771 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fcea0d-e47b-4aef-8e5d-a6849a2034a4 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:09.113599 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4588f20e-3962-4240-992d-a2dc73e2cbbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:09.118453 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588f20e-3962-4240-992d-a2dc73e2cbbb admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:09.119458 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588f20e-3962-4240-992d-a2dc73e2cbbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:09.119996 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588f20e-3962-4240-992d-a2dc73e2cbbb admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:09.120980 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4588f20e-3962-4240-992d-a2dc73e2cbbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:09.123924 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4588f20e-3962-4240-992d-a2dc73e2cbbb admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:09.173155 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1209/2445] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:47:09 2022] POST /identity/v3/users => generated 289 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:47:09.179126 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31242a4-2c54-457d-84f0-d2a388c006e8 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:09.202523 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31242a4-2c54-457d-84f0-d2a388c006e8 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:09.212390 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d5492a-f812-46e2-9037-6b44d6f6b1d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:09.214035 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d5492a-f812-46e2-9037-6b44d6f6b1d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:09.214320 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d5492a-f812-46e2-9037-6b44d6f6b1d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:09.214797 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d5492a-f812-46e2-9037-6b44d6f6b1d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:09.216579 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d5492a-f812-46e2-9037-6b44d6f6b1d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:09.220260 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d5492a-f812-46e2-9037-6b44d6f6b1d6 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:09.250140 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1237/2446] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:47:09 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:47:09.256371 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4588f20e-3962-4240-992d-a2dc73e2cbbb admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:09.281253 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4588f20e-3962-4240-992d-a2dc73e2cbbb admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:09.291957 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197b3d2a-8982-4354-9ec7-7cd7d6ee0b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:09.293145 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b3d2a-8982-4354-9ec7-7cd7d6ee0b64 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:09.293642 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b3d2a-8982-4354-9ec7-7cd7d6ee0b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:09.294193 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b3d2a-8982-4354-9ec7-7cd7d6ee0b64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:09.296548 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197b3d2a-8982-4354-9ec7-7cd7d6ee0b64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:09.300907 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197b3d2a-8982-4354-9ec7-7cd7d6ee0b64 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:09.326376 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1210/2447] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:47:09 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:47:09.331334 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d5492a-f812-46e2-9037-6b44d6f6b1d6 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:09.354201 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d5492a-f812-46e2-9037-6b44d6f6b1d6 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:09.370035 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b13760d-90cf-460b-bf7a-cf1f5dfbc1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:09.371774 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13760d-90cf-460b-bf7a-cf1f5dfbc1b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:09.373597 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13760d-90cf-460b-bf7a-cf1f5dfbc1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:09.374283 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13760d-90cf-460b-bf7a-cf1f5dfbc1b3 admin admin] PATH_INFO: `/v3/projects/d4ed6d817ba6423aa4c984eee6022af9/users/4269221da71344a7b76a4a0498e0118f/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:09.415424 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b13760d-90cf-460b-bf7a-cf1f5dfbc1b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4ed6d817ba6423aa4c984eee6022af9, user_id=4269221da71344a7b76a4a0498e0118f, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:09.423482 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b13760d-90cf-460b-bf7a-cf1f5dfbc1b3 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:09.470058 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1238/2448] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:47:09 2022] PUT /identity/v3/projects/d4ed6d817ba6423aa4c984eee6022af9/users/4269221da71344a7b76a4a0498e0118f/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:47:09.496365 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61d985e-7961-4e45-8b5f-e6a2fef90fb7 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:09.497129 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61d985e-7961-4e45-8b5f-e6a2fef90fb7 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:09.498760 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61d985e-7961-4e45-8b5f-e6a2fef90fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:09.596040 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-c61d985e-7961-4e45-8b5f-e6a2fef90fb7 None None] MFA Rules not processed for user `4269221da71344a7b76a4a0498e0118f`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:47:09.598387 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-c61d985e-7961-4e45-8b5f-e6a2fef90fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:09.848922 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1211/2449] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:47:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:47:09.850712 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b13760d-90cf-460b-bf7a-cf1f5dfbc1b3 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:10.053728 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b13760d-90cf-460b-bf7a-cf1f5dfbc1b3 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:10.062664 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07502b7e-ebb8-4b0c-ac93-ce22a99a4f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:10.063950 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07502b7e-ebb8-4b0c-ac93-ce22a99a4f15 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:10.064567 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07502b7e-ebb8-4b0c-ac93-ce22a99a4f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:10.065176 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07502b7e-ebb8-4b0c-ac93-ce22a99a4f15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:10.066763 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07502b7e-ebb8-4b0c-ac93-ce22a99a4f15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:10.070503 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07502b7e-ebb8-4b0c-ac93-ce22a99a4f15 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:10.140037 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1239/2450] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:47:09 2022] POST /identity/v3/projects => generated 375 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:47:10.140481 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61d985e-7961-4e45-8b5f-e6a2fef90fb7 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:10.180734 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61d985e-7961-4e45-8b5f-e6a2fef90fb7 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:10.202773 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:47:10.359755 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6410e54f-c7d4-46f7-883e-d2d6747e4529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:10.360715 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410e54f-c7d4-46f7-883e-d2d6747e4529 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:10.361189 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410e54f-c7d4-46f7-883e-d2d6747e4529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:10.362205 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410e54f-c7d4-46f7-883e-d2d6747e4529 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:10.362915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6410e54f-c7d4-46f7-883e-d2d6747e4529 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:10.368282 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6410e54f-c7d4-46f7-883e-d2d6747e4529 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:10.421764 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1212/2451] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:47:10 2022] POST /identity/v3/users => generated 294 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:47:10.425854 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07502b7e-ebb8-4b0c-ac93-ce22a99a4f15 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:10.440220 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:47:10.448112 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07502b7e-ebb8-4b0c-ac93-ce22a99a4f15 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:10.459958 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2c02a2-7534-4294-96a9-8ef42cb2d917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:10.462268 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2c02a2-7534-4294-96a9-8ef42cb2d917 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:10.462919 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2c02a2-7534-4294-96a9-8ef42cb2d917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:10.463772 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2c02a2-7534-4294-96a9-8ef42cb2d917 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:10.464804 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2c02a2-7534-4294-96a9-8ef42cb2d917 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:10.469101 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2c02a2-7534-4294-96a9-8ef42cb2d917 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:10.504596 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1240/2452] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:47:10 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:47:10.510981 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6410e54f-c7d4-46f7-883e-d2d6747e4529 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:10.535363 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6410e54f-c7d4-46f7-883e-d2d6747e4529 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:10.548550 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630afdc3-3569-4fbc-a94a-beef376fca3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:10.550182 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630afdc3-3569-4fbc-a94a-beef376fca3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:10.550691 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630afdc3-3569-4fbc-a94a-beef376fca3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:10.551181 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630afdc3-3569-4fbc-a94a-beef376fca3e admin admin] PATH_INFO: `/v3/projects/129ca20d71fd47388006ac40397fd302/users/4d1a5bb8c1a942068826a75ad99477f7/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:10.605642 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630afdc3-3569-4fbc-a94a-beef376fca3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=129ca20d71fd47388006ac40397fd302, user_id=4d1a5bb8c1a942068826a75ad99477f7, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:10.617719 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630afdc3-3569-4fbc-a94a-beef376fca3e admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:10.665310 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1213/2453] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:47:10 2022] PUT /identity/v3/projects/129ca20d71fd47388006ac40397fd302/users/4d1a5bb8c1a942068826a75ad99477f7/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:47:10.666107 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2c02a2-7534-4294-96a9-8ef42cb2d917 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:10.700047 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2c02a2-7534-4294-96a9-8ef42cb2d917 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:10.853331 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf7112f-2c59-4c93-acd9-1c63fedbcc99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:10.854577 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7112f-2c59-4c93-acd9-1c63fedbcc99 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:10.855079 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7112f-2c59-4c93-acd9-1c63fedbcc99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:10.855572 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7112f-2c59-4c93-acd9-1c63fedbcc99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:10.858393 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf7112f-2c59-4c93-acd9-1c63fedbcc99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:10.864094 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf7112f-2c59-4c93-acd9-1c63fedbcc99 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:10.893275 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1241/2454] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:47:10 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:47:10.895150 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630afdc3-3569-4fbc-a94a-beef376fca3e admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:10.922536 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630afdc3-3569-4fbc-a94a-beef376fca3e admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:10.931574 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2930d9c0-2d8f-4a87-8090-76828e8a9439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:10.937483 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2930d9c0-2d8f-4a87-8090-76828e8a9439 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:10.937483 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2930d9c0-2d8f-4a87-8090-76828e8a9439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:10.937483 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2930d9c0-2d8f-4a87-8090-76828e8a9439 admin admin] PATH_INFO: `/v3/projects/129ca20d71fd47388006ac40397fd302/users/4d1a5bb8c1a942068826a75ad99477f7/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:10.941393 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2930d9c0-2d8f-4a87-8090-76828e8a9439 admin admin] RBAC: Authorizing `identity:create_grant(project_id=129ca20d71fd47388006ac40397fd302, user_id=4d1a5bb8c1a942068826a75ad99477f7, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:10.947019 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2930d9c0-2d8f-4a87-8090-76828e8a9439 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:10.970426 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1214/2455] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:47:10 2022] PUT /identity/v3/projects/129ca20d71fd47388006ac40397fd302/users/4d1a5bb8c1a942068826a75ad99477f7/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:47:10.983966 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4046ef-63fb-4a2a-9c0c-6f10a751560a None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:10.984300 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4046ef-63fb-4a2a-9c0c-6f10a751560a None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:10.984495 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4046ef-63fb-4a2a-9c0c-6f10a751560a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:11.030000 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-0b4046ef-63fb-4a2a-9c0c-6f10a751560a None None] MFA Rules not processed for user `4d1a5bb8c1a942068826a75ad99477f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:47:11.034069 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-0b4046ef-63fb-4a2a-9c0c-6f10a751560a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:11.260151 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1242/2456] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:47:10 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:47:11.316005 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2930d9c0-2d8f-4a87-8090-76828e8a9439 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:11.319247 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-2930d9c0-2d8f-4a87-8090-76828e8a9439 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:11.604974 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2930d9c0-2d8f-4a87-8090-76828e8a9439 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:11.615096 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b2419-37b0-4c1e-b0e8-0ef60178ed30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:11.617068 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b2419-37b0-4c1e-b0e8-0ef60178ed30 service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:11.617983 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b2419-37b0-4c1e-b0e8-0ef60178ed30 service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:11.618723 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b2419-37b0-4c1e-b0e8-0ef60178ed30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:11.829553 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7b2419-37b0-4c1e-b0e8-0ef60178ed30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:11.833294 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7b2419-37b0-4c1e-b0e8-0ef60178ed30 service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:11.861995 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1215/2457] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:47:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:47:12.449445 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd602a5-1ddb-483c-8434-591c921490d9 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:12.450274 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd602a5-1ddb-483c-8434-591c921490d9 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:12.451186 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd602a5-1ddb-483c-8434-591c921490d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:12.527019 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-9dd602a5-1ddb-483c-8434-591c921490d9 None None] MFA Rules not processed for user `85d9c4c57681447c898ebfd7990b05b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:47:12.534001 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-9dd602a5-1ddb-483c-8434-591c921490d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:12.792824 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1243/2458] 10.222.0.13 () {58 vars in 1009 bytes} [Fri Jun 24 08:47:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:47:12.935072 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b2419-37b0-4c1e-b0e8-0ef60178ed30 service cinder] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:12.943075 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-cd7b2419-37b0-4c1e-b0e8-0ef60178ed30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:12.979887 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:47:13.096061 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b2419-37b0-4c1e-b0e8-0ef60178ed30 service cinder] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:13.101839 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54ed810-c634-4147-8ea4-ec0658a1cd5a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec8c2f113cf41a0939b81f38fbee813', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:13.103150 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54ed810-c634-4147-8ea4-ec0658a1cd5a service swift] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:13.103150 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54ed810-c634-4147-8ea4-ec0658a1cd5a service swift] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:13.103150 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54ed810-c634-4147-8ea4-ec0658a1cd5a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:13.140965 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54ed810-c634-4147-8ea4-ec0658a1cd5a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:13.142948 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54ed810-c634-4147-8ea4-ec0658a1cd5a service swift] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:13.180560 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1216/2459] 10.222.0.13 () {60 vars in 1384 bytes} [Fri Jun 24 08:47:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 08:47:14.527870 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae519b5-51ef-4bac-b3fb-ee55a08a0b7d None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:14.528669 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae519b5-51ef-4bac-b3fb-ee55a08a0b7d None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:14.530846 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae519b5-51ef-4bac-b3fb-ee55a08a0b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:14.647047 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-fae519b5-51ef-4bac-b3fb-ee55a08a0b7d None None] MFA Rules not processed for user `8d15a76041274e7381087bd21eb674c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:47:14.648649 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-fae519b5-51ef-4bac-b3fb-ee55a08a0b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:14.667934 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:47:14.796019 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1244/2460] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:47:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:47:14.811788 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54ed810-c634-4147-8ea4-ec0658a1cd5a service swift] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:14.813403 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-f54ed810-c634-4147-8ea4-ec0658a1cd5a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:14.980597 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54ed810-c634-4147-8ea4-ec0658a1cd5a service swift] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:14.993730 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af98bc4b-7e76-441f-a601-dbdff6c3d64e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74bb1bc6821b4500b26f160ccae5b7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:14.995025 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af98bc4b-7e76-441f-a601-dbdff6c3d64e service nova] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:14.995749 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af98bc4b-7e76-441f-a601-dbdff6c3d64e service nova] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:14.996110 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af98bc4b-7e76-441f-a601-dbdff6c3d64e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:15.008565 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af98bc4b-7e76-441f-a601-dbdff6c3d64e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:15.012932 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af98bc4b-7e76-441f-a601-dbdff6c3d64e service nova] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:15.025613 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1217/2461] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:47:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:47:17.784629 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae519b5-51ef-4bac-b3fb-ee55a08a0b7d None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:17.804237 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae519b5-51ef-4bac-b3fb-ee55a08a0b7d None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:17.818521 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acabb6eb-fbc6-4aea-9a24-1fb7f1b84c78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:17.820528 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acabb6eb-fbc6-4aea-9a24-1fb7f1b84c78 service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:17.821045 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acabb6eb-fbc6-4aea-9a24-1fb7f1b84c78 service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:17.822354 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acabb6eb-fbc6-4aea-9a24-1fb7f1b84c78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:17.845236 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acabb6eb-fbc6-4aea-9a24-1fb7f1b84c78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:17.848356 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acabb6eb-fbc6-4aea-9a24-1fb7f1b84c78 service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:17.877220 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6edb89d-d574-4802-86d6-5eefc1224cdd None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:17.877830 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6edb89d-d574-4802-86d6-5eefc1224cdd None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:17.878455 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6edb89d-d574-4802-86d6-5eefc1224cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:17.883420 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1245/2462] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:47:17 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:47:17.940131 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-d6edb89d-d574-4802-86d6-5eefc1224cdd None None] MFA Rules not processed for user `8d15a76041274e7381087bd21eb674c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:47:17.942498 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d6edb89d-d574-4802-86d6-5eefc1224cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:17.953413 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1218/2463] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:47:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:47:18.726746 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acabb6eb-fbc6-4aea-9a24-1fb7f1b84c78 service cinder] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:18.729294 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-acabb6eb-fbc6-4aea-9a24-1fb7f1b84c78 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:18.977144 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acabb6eb-fbc6-4aea-9a24-1fb7f1b84c78 service cinder] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:18.983394 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0159c8e3-5d0f-4ef0-aab6-1312d386dd29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:18.986357 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0159c8e3-5d0f-4ef0-aab6-1312d386dd29 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:18.988488 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0159c8e3-5d0f-4ef0-aab6-1312d386dd29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:18.989584 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0159c8e3-5d0f-4ef0-aab6-1312d386dd29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:18.996687 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-0159c8e3-5d0f-4ef0-aab6-1312d386dd29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:19.020503 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0159c8e3-5d0f-4ef0-aab6-1312d386dd29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:19.022741 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0159c8e3-5d0f-4ef0-aab6-1312d386dd29 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:19.036069 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1246/2464] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:47:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:47:27.909892 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=62 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:47:29.033243 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=56 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:47:36.054471 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df6b23b-d217-43a7-b0c4-72dd15c2c38b None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:36.055320 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df6b23b-d217-43a7-b0c4-72dd15c2c38b None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:36.055480 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df6b23b-d217-43a7-b0c4-72dd15c2c38b None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:36.057144 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1219/2465] 10.222.0.13 () {54 vars in 921 bytes} [Fri Jun 24 08:47:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:47:36.066893 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29861e1c-9622-4d44-83b2-d48d2f2042e0 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:36.067736 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29861e1c-9622-4d44-83b2-d48d2f2042e0 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:36.067929 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29861e1c-9622-4d44-83b2-d48d2f2042e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:36.089935 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-29861e1c-9622-4d44-83b2-d48d2f2042e0 None None] MFA Rules not processed for user `74bb1bc6821b4500b26f160ccae5b7d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:47:36.092570 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-29861e1c-9622-4d44-83b2-d48d2f2042e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:36.104231 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1247/2466] 10.222.0.13 () {58 vars in 1018 bytes} [Fri Jun 24 08:47:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:47:36.119759 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df6b23b-d217-43a7-b0c4-72dd15c2c38b None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:36.121791 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-5df6b23b-d217-43a7-b0c4-72dd15c2c38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:36.240997 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df6b23b-d217-43a7-b0c4-72dd15c2c38b None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:36.244849 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7717d45d-a18e-46a6-9a82-9bcc63cb8267 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd15c5f6aeb874191b73c8ee0410c6bc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:36.245508 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7717d45d-a18e-46a6-9a82-9bcc63cb8267 service glance] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:36.245832 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7717d45d-a18e-46a6-9a82-9bcc63cb8267 service glance] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:36.246112 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7717d45d-a18e-46a6-9a82-9bcc63cb8267 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:36.257834 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7717d45d-a18e-46a6-9a82-9bcc63cb8267 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:36.259294 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7717d45d-a18e-46a6-9a82-9bcc63cb8267 service glance] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:36.271927 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1220/2467] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:47:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:47:36.342391 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd0b0c-b07c-4a56-b831-5ee26e056b82 None None] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:36.342684 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd0b0c-b07c-4a56-b831-5ee26e056b82 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:36.342955 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd0b0c-b07c-4a56-b831-5ee26e056b82 None None] PATH_INFO: `/` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:36.344381 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1248/2468] 10.222.0.13 () {54 vars in 921 bytes} [Fri Jun 24 08:47:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:47:36.349765 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b3ad1-efa4-400f-b163-84aaeb8c31a2 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:36.350058 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b3ad1-efa4-400f-b163-84aaeb8c31a2 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:36.350344 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b3ad1-efa4-400f-b163-84aaeb8c31a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:36.370898 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-147b3ad1-efa4-400f-b163-84aaeb8c31a2 None None] MFA Rules not processed for user `c5f7ba6c7faf43e7acbb43eee58d8c8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:47:36.372581 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-147b3ad1-efa4-400f-b163-84aaeb8c31a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:36.381387 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1221/2469] 10.222.0.13 () {58 vars in 1018 bytes} [Fri Jun 24 08:47:36 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:47:36.393769 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd0b0c-b07c-4a56-b831-5ee26e056b82 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:36.406510 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd0b0c-b07c-4a56-b831-5ee26e056b82 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:36.410831 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f237070-474d-4e07-ac18-a1dc73c2447c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:36.411557 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f237070-474d-4e07-ac18-a1dc73c2447c service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:36.411918 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f237070-474d-4e07-ac18-a1dc73c2447c service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:36.412236 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f237070-474d-4e07-ac18-a1dc73c2447c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:36.421902 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f237070-474d-4e07-ac18-a1dc73c2447c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:36.423536 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f237070-474d-4e07-ac18-a1dc73c2447c service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:36.432774 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1249/2470] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:47:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:47:36.526325 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f562af86-3367-41c7-954d-f135050cda94 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:36.526690 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f562af86-3367-41c7-954d-f135050cda94 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:36.527010 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f562af86-3367-41c7-954d-f135050cda94 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:36.529005 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1222/2471] 10.222.0.13 () {54 vars in 921 bytes} [Fri Jun 24 08:47:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:47:36.535504 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f10327-175d-445a-a379-97d7b6680213 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:36.535853 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f10327-175d-445a-a379-97d7b6680213 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:36.536228 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f10327-175d-445a-a379-97d7b6680213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:36.558138 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-87f10327-175d-445a-a379-97d7b6680213 None None] MFA Rules not processed for user `74bb1bc6821b4500b26f160ccae5b7d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:47:36.560414 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-87f10327-175d-445a-a379-97d7b6680213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:36.571179 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1250/2472] 10.222.0.13 () {58 vars in 1018 bytes} [Fri Jun 24 08:47:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:47:36.582276 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f562af86-3367-41c7-954d-f135050cda94 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:36.594927 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f562af86-3367-41c7-954d-f135050cda94 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:36.599202 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4c894b-3aae-4be4-b9a4-048dbc0e398e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:36.599887 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c894b-3aae-4be4-b9a4-048dbc0e398e service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:36.600205 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c894b-3aae-4be4-b9a4-048dbc0e398e service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:36.600523 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c894b-3aae-4be4-b9a4-048dbc0e398e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:36.612379 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4c894b-3aae-4be4-b9a4-048dbc0e398e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:36.613855 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4c894b-3aae-4be4-b9a4-048dbc0e398e service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:36.626006 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1223/2473] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:47:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:47:40.336904 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b229a-4997-4820-b534-f98640c50ec7 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:40.337745 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b229a-4997-4820-b534-f98640c50ec7 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:40.337745 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b229a-4997-4820-b534-f98640c50ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:40.356945 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-7d3b229a-4997-4820-b534-f98640c50ec7 None None] MFA Rules not processed for user `85d9c4c57681447c898ebfd7990b05b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:47:40.358457 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-7d3b229a-4997-4820-b534-f98640c50ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:40.366201 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1251/2474] 10.222.0.13 () {58 vars in 1009 bytes} [Fri Jun 24 08:47:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:47:40.382774 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4c894b-3aae-4be4-b9a4-048dbc0e398e service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:40.395770 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4c894b-3aae-4be4-b9a4-048dbc0e398e service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:40.400868 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c3026a-bb5a-482d-b40f-97d606be271d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec8c2f113cf41a0939b81f38fbee813', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:40.401549 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c3026a-bb5a-482d-b40f-97d606be271d service swift] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:40.401872 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c3026a-bb5a-482d-b40f-97d606be271d service swift] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:40.402154 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c3026a-bb5a-482d-b40f-97d606be271d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:40.411979 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c3026a-bb5a-482d-b40f-97d606be271d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:40.413637 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c3026a-bb5a-482d-b40f-97d606be271d service swift] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:40.422791 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1224/2475] 10.222.0.13 () {60 vars in 1384 bytes} [Fri Jun 24 08:47:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 08:47:44.303701 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3b229a-4997-4820-b534-f98640c50ec7 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:44.399099 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3b229a-4997-4820-b534-f98640c50ec7 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:44.406392 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c743c0-42fa-4634-8e2c-371ad6ebcae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:44.407856 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c743c0-42fa-4634-8e2c-371ad6ebcae1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:44.408167 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c743c0-42fa-4634-8e2c-371ad6ebcae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:44.408623 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c743c0-42fa-4634-8e2c-371ad6ebcae1 admin admin] PATH_INFO: `/v3/users/4269221da71344a7b76a4a0498e0118f` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:44.410518 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c743c0-42fa-4634-8e2c-371ad6ebcae1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4269221da71344a7b76a4a0498e0118f)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:44.413386 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c743c0-42fa-4634-8e2c-371ad6ebcae1 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:44.492372 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-29c743c0-42fa-4634-8e2c-371ad6ebcae1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4269221da71344a7b76a4a0498e0118f'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:44.509421 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-29c743c0-42fa-4634-8e2c-371ad6ebcae1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4269221da71344a7b76a4a0498e0118f'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:44.520422 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-29c743c0-42fa-4634-8e2c-371ad6ebcae1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4269221da71344a7b76a4a0498e0118f'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:44.521439 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-29c743c0-42fa-4634-8e2c-371ad6ebcae1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4269221da71344a7b76a4a0498e0118f'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:44.523257 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-29c743c0-42fa-4634-8e2c-371ad6ebcae1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4269221da71344a7b76a4a0498e0118f'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:44.548958 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1252/2476] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:47:44 2022] DELETE /identity/v3/users/4269221da71344a7b76a4a0498e0118f => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:47:44.553959 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3bc25c-743d-4511-a581-8511f80028df None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:44.554522 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3bc25c-743d-4511-a581-8511f80028df None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:44.554522 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3bc25c-743d-4511-a581-8511f80028df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:44.582686 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-9a3bc25c-743d-4511-a581-8511f80028df None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:47:44.583947 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-9a3bc25c-743d-4511-a581-8511f80028df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:44.665040 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1225/2477] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:47:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:47:44.677957 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c743c0-42fa-4634-8e2c-371ad6ebcae1 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:44.681060 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-29c743c0-42fa-4634-8e2c-371ad6ebcae1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:44.747859 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:47:44.760359 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c743c0-42fa-4634-8e2c-371ad6ebcae1 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:44.774036 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9672c76d-22dc-4b34-9801-424d22d6643d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:44.775328 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9672c76d-22dc-4b34-9801-424d22d6643d service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:44.776128 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9672c76d-22dc-4b34-9801-424d22d6643d service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:44.777041 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9672c76d-22dc-4b34-9801-424d22d6643d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:44.796619 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9672c76d-22dc-4b34-9801-424d22d6643d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:44.798435 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9672c76d-22dc-4b34-9801-424d22d6643d service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:44.811745 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1253/2478] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:47:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:47:45.374505 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3bc25c-743d-4511-a581-8511f80028df None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:45.376117 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-9a3bc25c-743d-4511-a581-8511f80028df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:45.399959 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3bc25c-743d-4511-a581-8511f80028df None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:45.419750 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b10c2c-4b7d-4fda-83dd-6f9f44eb462c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:45.423653 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b10c2c-4b7d-4fda-83dd-6f9f44eb462c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:45.424159 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b10c2c-4b7d-4fda-83dd-6f9f44eb462c admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:45.424745 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b10c2c-4b7d-4fda-83dd-6f9f44eb462c admin admin] PATH_INFO: `/v3/projects/d4ed6d817ba6423aa4c984eee6022af9` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:45.428021 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b10c2c-4b7d-4fda-83dd-6f9f44eb462c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4ed6d817ba6423aa4c984eee6022af9)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:45.432624 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b10c2c-4b7d-4fda-83dd-6f9f44eb462c admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:45.547011 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:47:45.554814 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-e2b10c2c-4b7d-4fda-83dd-6f9f44eb462c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4ed6d817ba6423aa4c984eee6022af9', 'request_id': 'req-e2b10c2c-4b7d-4fda-83dd-6f9f44eb462c'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:45.567889 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-e2b10c2c-4b7d-4fda-83dd-6f9f44eb462c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4ed6d817ba6423aa4c984eee6022af9', 'request_id': 'req-e2b10c2c-4b7d-4fda-83dd-6f9f44eb462c'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:45.578766 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-e2b10c2c-4b7d-4fda-83dd-6f9f44eb462c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4ed6d817ba6423aa4c984eee6022af9', 'request_id': 'req-e2b10c2c-4b7d-4fda-83dd-6f9f44eb462c'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:45.607081 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-e2b10c2c-4b7d-4fda-83dd-6f9f44eb462c admin admin] The token cache is being invalidate because project d4ed6d817ba6423aa4c984eee6022af9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:47:45.607679 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-e2b10c2c-4b7d-4fda-83dd-6f9f44eb462c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:45.608826 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-e2b10c2c-4b7d-4fda-83dd-6f9f44eb462c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:45.611860 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1226/2479] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:47:45 2022] DELETE /identity/v3/projects/d4ed6d817ba6423aa4c984eee6022af9 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:47:45.618598 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9672c76d-22dc-4b34-9801-424d22d6643d service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:45.638208 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9672c76d-22dc-4b34-9801-424d22d6643d service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:45.733997 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed01df2-a2ed-47e6-9257-f05156201097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:45.735514 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed01df2-a2ed-47e6-9257-f05156201097 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:45.736030 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed01df2-a2ed-47e6-9257-f05156201097 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:45.736537 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed01df2-a2ed-47e6-9257-f05156201097 admin admin] PATH_INFO: `/v3/users/4d1a5bb8c1a942068826a75ad99477f7` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:45.738851 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed01df2-a2ed-47e6-9257-f05156201097 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d1a5bb8c1a942068826a75ad99477f7)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:45.742684 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed01df2-a2ed-47e6-9257-f05156201097 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:45.813296 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-2ed01df2-a2ed-47e6-9257-f05156201097 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d1a5bb8c1a942068826a75ad99477f7'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:45.837399 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-2ed01df2-a2ed-47e6-9257-f05156201097 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d1a5bb8c1a942068826a75ad99477f7'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:45.848638 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-2ed01df2-a2ed-47e6-9257-f05156201097 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d1a5bb8c1a942068826a75ad99477f7'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:45.849661 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-2ed01df2-a2ed-47e6-9257-f05156201097 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d1a5bb8c1a942068826a75ad99477f7'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:45.850657 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-2ed01df2-a2ed-47e6-9257-f05156201097 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d1a5bb8c1a942068826a75ad99477f7'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:45.876409 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1254/2480] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:47:45 2022] DELETE /identity/v3/users/4d1a5bb8c1a942068826a75ad99477f7 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:47:46.504062 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b10c2c-4b7d-4fda-83dd-6f9f44eb462c admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:46.504969 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-e2b10c2c-4b7d-4fda-83dd-6f9f44eb462c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:46.550230 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b10c2c-4b7d-4fda-83dd-6f9f44eb462c admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:46.743642 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e43d4-0d0b-4266-8080-1d3373431289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:46.744839 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e43d4-0d0b-4266-8080-1d3373431289 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:46.744839 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e43d4-0d0b-4266-8080-1d3373431289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:46.745149 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e43d4-0d0b-4266-8080-1d3373431289 admin admin] PATH_INFO: `/v3/projects/129ca20d71fd47388006ac40397fd302` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:46.746541 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8e43d4-0d0b-4266-8080-1d3373431289 admin admin] RBAC: Authorizing `identity:delete_project(project_id=129ca20d71fd47388006ac40397fd302)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:46.756832 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8e43d4-0d0b-4266-8080-1d3373431289 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:46.848084 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-de8e43d4-0d0b-4266-8080-1d3373431289 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '129ca20d71fd47388006ac40397fd302', 'request_id': 'req-de8e43d4-0d0b-4266-8080-1d3373431289'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:46.855949 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-de8e43d4-0d0b-4266-8080-1d3373431289 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '129ca20d71fd47388006ac40397fd302', 'request_id': 'req-de8e43d4-0d0b-4266-8080-1d3373431289'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:46.864291 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-de8e43d4-0d0b-4266-8080-1d3373431289 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '129ca20d71fd47388006ac40397fd302', 'request_id': 'req-de8e43d4-0d0b-4266-8080-1d3373431289'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:46.887397 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-de8e43d4-0d0b-4266-8080-1d3373431289 admin admin] The token cache is being invalidate because project 129ca20d71fd47388006ac40397fd302 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:47:46.887971 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-de8e43d4-0d0b-4266-8080-1d3373431289 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:46.889075 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-de8e43d4-0d0b-4266-8080-1d3373431289 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:46.897837 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1227/2481] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:47:46 2022] DELETE /identity/v3/projects/129ca20d71fd47388006ac40397fd302 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:47:46.900233 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcc1813-0e22-48e4-afd8-c3b4d906e640 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:46.901540 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcc1813-0e22-48e4-afd8-c3b4d906e640 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:46.902077 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcc1813-0e22-48e4-afd8-c3b4d906e640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:46.950821 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-4bcc1813-0e22-48e4-afd8-c3b4d906e640 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:47:46.953236 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-4bcc1813-0e22-48e4-afd8-c3b4d906e640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:47.104668 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1255/2482] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:47:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:47:47.114951 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1602a8-f3d7-499b-b3dd-15977ee7ea24 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:47.115332 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1602a8-f3d7-499b-b3dd-15977ee7ea24 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:47.115611 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1602a8-f3d7-499b-b3dd-15977ee7ea24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:47.136811 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-5e1602a8-f3d7-499b-b3dd-15977ee7ea24 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:47:47.138679 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-5e1602a8-f3d7-499b-b3dd-15977ee7ea24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:47.147956 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1228/2483] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:47:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:47:47.153950 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcc1813-0e22-48e4-afd8-c3b4d906e640 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:47.175079 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcc1813-0e22-48e4-afd8-c3b4d906e640 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:47.182976 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bfdbbe-93c1-4857-a1a3-42342419960a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:47.184057 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bfdbbe-93c1-4857-a1a3-42342419960a admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:47.184772 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bfdbbe-93c1-4857-a1a3-42342419960a admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:47.185635 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bfdbbe-93c1-4857-a1a3-42342419960a admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:47.186422 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bfdbbe-93c1-4857-a1a3-42342419960a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:47:47.186942 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bfdbbe-93c1-4857-a1a3-42342419960a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:47.190179 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bfdbbe-93c1-4857-a1a3-42342419960a admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:47.194786 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:47:47.214187 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1256/2484] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:47:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:47:47.221357 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1602a8-f3d7-499b-b3dd-15977ee7ea24 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:47.236498 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1602a8-f3d7-499b-b3dd-15977ee7ea24 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:47.242783 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41f53e0-3e00-4546-8c4e-76509beb6d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:47.243789 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f53e0-3e00-4546-8c4e-76509beb6d9a admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:47.244251 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f53e0-3e00-4546-8c4e-76509beb6d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:47.244688 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f53e0-3e00-4546-8c4e-76509beb6d9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:47.245571 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41f53e0-3e00-4546-8c4e-76509beb6d9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:47.248348 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41f53e0-3e00-4546-8c4e-76509beb6d9a admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:47.290629 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1229/2485] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:47:47 2022] POST /identity/v3/projects => generated 383 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:47:47.295778 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bfdbbe-93c1-4857-a1a3-42342419960a admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:47.322320 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bfdbbe-93c1-4857-a1a3-42342419960a admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:47.508901 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6251096b-9413-4537-a933-f321220d49c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:47.512107 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251096b-9413-4537-a933-f321220d49c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:47.512948 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251096b-9413-4537-a933-f321220d49c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:47.513333 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251096b-9413-4537-a933-f321220d49c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:47.514509 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6251096b-9413-4537-a933-f321220d49c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:47.517727 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6251096b-9413-4537-a933-f321220d49c8 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:47.574162 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1257/2486] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:47:47 2022] POST /identity/v3/users => generated 292 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:47:47.580324 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41f53e0-3e00-4546-8c4e-76509beb6d9a admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:47.600528 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41f53e0-3e00-4546-8c4e-76509beb6d9a admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:47.610035 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22f8794-36d6-4f20-abbf-1fb8ea42fcdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:47.611324 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f8794-36d6-4f20-abbf-1fb8ea42fcdd admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:47.611916 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f8794-36d6-4f20-abbf-1fb8ea42fcdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:47.612493 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f8794-36d6-4f20-abbf-1fb8ea42fcdd admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:47.614265 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22f8794-36d6-4f20-abbf-1fb8ea42fcdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:47.617458 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22f8794-36d6-4f20-abbf-1fb8ea42fcdd admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:47.640164 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1230/2487] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:47:47 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:47:47.641427 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6251096b-9413-4537-a933-f321220d49c8 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:47.668254 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6251096b-9413-4537-a933-f321220d49c8 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:47.675484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfda206-e642-4a08-8731-70237a805f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:47.676249 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfda206-e642-4a08-8731-70237a805f80 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:47.676583 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfda206-e642-4a08-8731-70237a805f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:47.676864 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfda206-e642-4a08-8731-70237a805f80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:47.677837 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfda206-e642-4a08-8731-70237a805f80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:47.679764 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfda206-e642-4a08-8731-70237a805f80 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:47.692695 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1258/2488] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:47:47 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:47:47.696957 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22f8794-36d6-4f20-abbf-1fb8ea42fcdd admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:47.708148 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22f8794-36d6-4f20-abbf-1fb8ea42fcdd admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:47.713017 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e3feae-6467-4559-b731-8ee262d5e219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:47.713958 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3feae-6467-4559-b731-8ee262d5e219 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:47.714246 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3feae-6467-4559-b731-8ee262d5e219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:47.714521 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3feae-6467-4559-b731-8ee262d5e219 admin admin] PATH_INFO: `/v3/projects/2edabce64b3c4b60ac84971773ba948f/users/5d302e78ecda420dbf14905a6f60beca/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:47.733728 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e3feae-6467-4559-b731-8ee262d5e219 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2edabce64b3c4b60ac84971773ba948f, user_id=5d302e78ecda420dbf14905a6f60beca, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:47.736359 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e3feae-6467-4559-b731-8ee262d5e219 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:47.767271 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:47:47.784826 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1231/2489] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:47:47 2022] PUT /identity/v3/projects/2edabce64b3c4b60ac84971773ba948f/users/5d302e78ecda420dbf14905a6f60beca/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:47:47.788237 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c5a4ea-9a3e-40de-ac67-92ab59fb9d68 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:47.788842 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c5a4ea-9a3e-40de-ac67-92ab59fb9d68 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:47.789390 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c5a4ea-9a3e-40de-ac67-92ab59fb9d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:47.846173 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-f1c5a4ea-9a3e-40de-ac67-92ab59fb9d68 None None] MFA Rules not processed for user `5d302e78ecda420dbf14905a6f60beca`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:47:47.847497 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-f1c5a4ea-9a3e-40de-ac67-92ab59fb9d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:47.966594 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1259/2490] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:47:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:47:47.971062 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e3feae-6467-4559-b731-8ee262d5e219 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:48.103139 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e3feae-6467-4559-b731-8ee262d5e219 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:48.118327 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e35e6-df57-4843-8065-e0d4ddda9337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:48.119559 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e35e6-df57-4843-8065-e0d4ddda9337 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:48.120133 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e35e6-df57-4843-8065-e0d4ddda9337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:48.120741 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e35e6-df57-4843-8065-e0d4ddda9337 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:48.122110 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e35e6-df57-4843-8065-e0d4ddda9337 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:48.125744 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e35e6-df57-4843-8065-e0d4ddda9337 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:48.197874 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1232/2491] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:47:47 2022] POST /identity/v3/projects => generated 381 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:47:48.203080 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c5a4ea-9a3e-40de-ac67-92ab59fb9d68 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:48.229776 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c5a4ea-9a3e-40de-ac67-92ab59fb9d68 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:48.364821 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e094c0-9517-4d89-ab1f-2dee8315b996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:48.365626 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e094c0-9517-4d89-ab1f-2dee8315b996 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:48.365956 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e094c0-9517-4d89-ab1f-2dee8315b996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:48.366363 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e094c0-9517-4d89-ab1f-2dee8315b996 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:48.370480 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e094c0-9517-4d89-ab1f-2dee8315b996 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:48.379974 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e094c0-9517-4d89-ab1f-2dee8315b996 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:48.396908 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:47:48.422790 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1260/2492] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:47:48 2022] POST /identity/v3/users => generated 297 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:47:48.429995 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e35e6-df57-4843-8065-e0d4ddda9337 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:48.449657 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e35e6-df57-4843-8065-e0d4ddda9337 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:48.458089 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1d4f80-7783-48b6-8285-8dc09dcdd466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:48.459229 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d4f80-7783-48b6-8285-8dc09dcdd466 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:48.459725 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d4f80-7783-48b6-8285-8dc09dcdd466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:48.460229 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d4f80-7783-48b6-8285-8dc09dcdd466 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:48.461547 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1d4f80-7783-48b6-8285-8dc09dcdd466 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:48.464699 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1d4f80-7783-48b6-8285-8dc09dcdd466 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:48.489258 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1233/2493] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:47:48 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:47:48.494149 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e094c0-9517-4d89-ab1f-2dee8315b996 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:48.506676 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e094c0-9517-4d89-ab1f-2dee8315b996 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:48.511653 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2de810d-654a-4f72-b7d0-09ec00a37918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:48.512602 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2de810d-654a-4f72-b7d0-09ec00a37918 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:48.513091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2de810d-654a-4f72-b7d0-09ec00a37918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:48.513559 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2de810d-654a-4f72-b7d0-09ec00a37918 admin admin] PATH_INFO: `/v3/projects/9cff077a047a45fbbe49aa0e18626e9e/users/3ae59a80023e495fb8ee9d466907264e/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:48.532412 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2de810d-654a-4f72-b7d0-09ec00a37918 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cff077a047a45fbbe49aa0e18626e9e, user_id=3ae59a80023e495fb8ee9d466907264e, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:48.536254 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2de810d-654a-4f72-b7d0-09ec00a37918 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:48.555816 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1261/2494] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:47:48 2022] PUT /identity/v3/projects/9cff077a047a45fbbe49aa0e18626e9e/users/3ae59a80023e495fb8ee9d466907264e/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:47:48.560606 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1d4f80-7783-48b6-8285-8dc09dcdd466 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:48.571546 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1d4f80-7783-48b6-8285-8dc09dcdd466 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:48.651262 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f6f5f-bcd6-48bf-be90-7a9ca21b2978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:48.651847 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f6f5f-bcd6-48bf-be90-7a9ca21b2978 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:48.652130 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f6f5f-bcd6-48bf-be90-7a9ca21b2978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:48.652404 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f6f5f-bcd6-48bf-be90-7a9ca21b2978 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:48.653400 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3f6f5f-bcd6-48bf-be90-7a9ca21b2978 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:48.655081 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3f6f5f-bcd6-48bf-be90-7a9ca21b2978 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:48.667744 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1234/2495] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:47:48 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:47:48.672115 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2de810d-654a-4f72-b7d0-09ec00a37918 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:48.682838 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2de810d-654a-4f72-b7d0-09ec00a37918 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:48.688622 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071aa8ca-0e5d-412b-a6cb-fcad7b5e5486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:48.689645 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071aa8ca-0e5d-412b-a6cb-fcad7b5e5486 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:48.690025 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071aa8ca-0e5d-412b-a6cb-fcad7b5e5486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:48.690368 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071aa8ca-0e5d-412b-a6cb-fcad7b5e5486 admin admin] PATH_INFO: `/v3/projects/9cff077a047a45fbbe49aa0e18626e9e/users/3ae59a80023e495fb8ee9d466907264e/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:48.692514 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071aa8ca-0e5d-412b-a6cb-fcad7b5e5486 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cff077a047a45fbbe49aa0e18626e9e, user_id=3ae59a80023e495fb8ee9d466907264e, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:48.695347 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071aa8ca-0e5d-412b-a6cb-fcad7b5e5486 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:48.708007 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1262/2496] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:47:48 2022] PUT /identity/v3/projects/9cff077a047a45fbbe49aa0e18626e9e/users/3ae59a80023e495fb8ee9d466907264e/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:47:48.722362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f78c5f-0c38-4d8c-b788-e4550f4f2634 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:48.722760 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f78c5f-0c38-4d8c-b788-e4550f4f2634 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:48.723091 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f78c5f-0c38-4d8c-b788-e4550f4f2634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:48.757284 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-75f78c5f-0c38-4d8c-b788-e4550f4f2634 None None] MFA Rules not processed for user `3ae59a80023e495fb8ee9d466907264e`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:47:48.760137 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-75f78c5f-0c38-4d8c-b788-e4550f4f2634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:48.848878 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:47:48.862338 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1235/2497] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:47:48 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:47:48.874659 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071aa8ca-0e5d-412b-a6cb-fcad7b5e5486 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:48.876570 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-071aa8ca-0e5d-412b-a6cb-fcad7b5e5486 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:48.944827 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071aa8ca-0e5d-412b-a6cb-fcad7b5e5486 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:48.949384 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26a71da-a36d-418b-8f69-fbec891ed106 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:48.950775 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26a71da-a36d-418b-8f69-fbec891ed106 service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:48.951166 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26a71da-a36d-418b-8f69-fbec891ed106 service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:48.951542 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26a71da-a36d-418b-8f69-fbec891ed106 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:48.962487 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26a71da-a36d-418b-8f69-fbec891ed106 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:48.964165 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26a71da-a36d-418b-8f69-fbec891ed106 service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:48.976298 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1263/2498] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:47:48 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:47:49.108241 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f78c5f-0c38-4d8c-b788-e4550f4f2634 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:49.119724 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f78c5f-0c38-4d8c-b788-e4550f4f2634 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:49.124683 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1ee214-95c3-452f-866e-97cba4c67dfd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:49.125643 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ee214-95c3-452f-866e-97cba4c67dfd service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:49.126128 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ee214-95c3-452f-866e-97cba4c67dfd service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:49.126592 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ee214-95c3-452f-866e-97cba4c67dfd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:49.225179 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1ee214-95c3-452f-866e-97cba4c67dfd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:49.228331 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1ee214-95c3-452f-866e-97cba4c67dfd service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:49.250008 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1236/2499] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:47:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:47:52.030938 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26a71da-a36d-418b-8f69-fbec891ed106 service cinder] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:52.104143 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26a71da-a36d-418b-8f69-fbec891ed106 service cinder] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:52.110642 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb8a4f-632c-4ce3-9d5c-8c044e85e53d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:52.111831 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb8a4f-632c-4ce3-9d5c-8c044e85e53d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:52.112326 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb8a4f-632c-4ce3-9d5c-8c044e85e53d admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:52.112629 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb8a4f-632c-4ce3-9d5c-8c044e85e53d admin admin] PATH_INFO: `/v3/users/5d302e78ecda420dbf14905a6f60beca` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:52.114045 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bb8a4f-632c-4ce3-9d5c-8c044e85e53d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d302e78ecda420dbf14905a6f60beca)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:52.116717 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bb8a4f-632c-4ce3-9d5c-8c044e85e53d admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:52.177621 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-83bb8a4f-632c-4ce3-9d5c-8c044e85e53d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d302e78ecda420dbf14905a6f60beca'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:52.189175 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:INSERT=3,keystone:SELECT=89,keystone:DELETE=8 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:47:52.191387 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-83bb8a4f-632c-4ce3-9d5c-8c044e85e53d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d302e78ecda420dbf14905a6f60beca'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:52.200348 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-83bb8a4f-632c-4ce3-9d5c-8c044e85e53d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d302e78ecda420dbf14905a6f60beca'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:52.201205 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-83bb8a4f-632c-4ce3-9d5c-8c044e85e53d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d302e78ecda420dbf14905a6f60beca'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:52.201882 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-83bb8a4f-632c-4ce3-9d5c-8c044e85e53d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d302e78ecda420dbf14905a6f60beca'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:52.216520 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1264/2500] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:47:52 2022] DELETE /identity/v3/users/5d302e78ecda420dbf14905a6f60beca => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:47:52.221464 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45aa5b-cc16-463c-89fd-435ecb20c1be None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:52.222016 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45aa5b-cc16-463c-89fd-435ecb20c1be None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:52.222120 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45aa5b-cc16-463c-89fd-435ecb20c1be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:52.242830 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-fc45aa5b-cc16-463c-89fd-435ecb20c1be None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:47:52.244096 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-fc45aa5b-cc16-463c-89fd-435ecb20c1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:52.318803 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1237/2501] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:47:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:47:52.329890 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb8a4f-632c-4ce3-9d5c-8c044e85e53d admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:52.331394 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-83bb8a4f-632c-4ce3-9d5c-8c044e85e53d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:52.415142 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb8a4f-632c-4ce3-9d5c-8c044e85e53d admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:52.420373 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ac41b5-2da2-43a7-a314-36eecc4d3565 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:52.421042 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac41b5-2da2-43a7-a314-36eecc4d3565 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:52.421355 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac41b5-2da2-43a7-a314-36eecc4d3565 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:52.421617 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac41b5-2da2-43a7-a314-36eecc4d3565 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:52.434483 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ac41b5-2da2-43a7-a314-36eecc4d3565 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:52.435999 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ac41b5-2da2-43a7-a314-36eecc4d3565 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:52.447388 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1265/2502] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:47:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:47:53.026353 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc45aa5b-cc16-463c-89fd-435ecb20c1be None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:53.030943 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-fc45aa5b-cc16-463c-89fd-435ecb20c1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:53.061831 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc45aa5b-cc16-463c-89fd-435ecb20c1be None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:53.070914 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62d581c-9f8d-454b-94a8-7bdb118cf80b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:53.072683 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d581c-9f8d-454b-94a8-7bdb118cf80b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:53.073281 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d581c-9f8d-454b-94a8-7bdb118cf80b admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:53.074016 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d581c-9f8d-454b-94a8-7bdb118cf80b admin admin] PATH_INFO: `/v3/projects/2edabce64b3c4b60ac84971773ba948f` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:53.076851 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62d581c-9f8d-454b-94a8-7bdb118cf80b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2edabce64b3c4b60ac84971773ba948f)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:53.086101 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62d581c-9f8d-454b-94a8-7bdb118cf80b admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:53.152456 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-e62d581c-9f8d-454b-94a8-7bdb118cf80b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2edabce64b3c4b60ac84971773ba948f', 'request_id': 'req-e62d581c-9f8d-454b-94a8-7bdb118cf80b'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:53.160312 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-e62d581c-9f8d-454b-94a8-7bdb118cf80b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2edabce64b3c4b60ac84971773ba948f', 'request_id': 'req-e62d581c-9f8d-454b-94a8-7bdb118cf80b'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:53.166706 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-e62d581c-9f8d-454b-94a8-7bdb118cf80b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2edabce64b3c4b60ac84971773ba948f', 'request_id': 'req-e62d581c-9f8d-454b-94a8-7bdb118cf80b'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:53.212725 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-e62d581c-9f8d-454b-94a8-7bdb118cf80b admin admin] The token cache is being invalidate because project 2edabce64b3c4b60ac84971773ba948f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:47:53.213426 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-e62d581c-9f8d-454b-94a8-7bdb118cf80b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:53.214708 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-e62d581c-9f8d-454b-94a8-7bdb118cf80b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:53.218096 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1238/2503] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:47:53 2022] DELETE /identity/v3/projects/2edabce64b3c4b60ac84971773ba948f => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:47:53.228313 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ac41b5-2da2-43a7-a314-36eecc4d3565 service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:53.246077 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ac41b5-2da2-43a7-a314-36eecc4d3565 service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:53.357032 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e081a900-c508-493c-9923-72982ac92b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:53.358091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081a900-c508-493c-9923-72982ac92b89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:53.358413 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081a900-c508-493c-9923-72982ac92b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:53.358654 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081a900-c508-493c-9923-72982ac92b89 admin admin] PATH_INFO: `/v3/users/3ae59a80023e495fb8ee9d466907264e` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:53.360129 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e081a900-c508-493c-9923-72982ac92b89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ae59a80023e495fb8ee9d466907264e)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:53.362293 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e081a900-c508-493c-9923-72982ac92b89 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:53.422376 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-e081a900-c508-493c-9923-72982ac92b89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ae59a80023e495fb8ee9d466907264e'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:53.440657 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-e081a900-c508-493c-9923-72982ac92b89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ae59a80023e495fb8ee9d466907264e'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:53.453392 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-e081a900-c508-493c-9923-72982ac92b89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ae59a80023e495fb8ee9d466907264e'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:53.454299 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-e081a900-c508-493c-9923-72982ac92b89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ae59a80023e495fb8ee9d466907264e'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:53.455099 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-e081a900-c508-493c-9923-72982ac92b89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ae59a80023e495fb8ee9d466907264e'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:53.473743 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:47:53.479374 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1266/2504] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:47:53 2022] DELETE /identity/v3/users/3ae59a80023e495fb8ee9d466907264e => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:47:53.827337 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62d581c-9f8d-454b-94a8-7bdb118cf80b admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:53.829865 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-e62d581c-9f8d-454b-94a8-7bdb118cf80b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:53.858399 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62d581c-9f8d-454b-94a8-7bdb118cf80b admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:53.884543 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:47:53.949667 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19414afd-0437-4272-a22b-bf82375ca8c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:53.952164 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19414afd-0437-4272-a22b-bf82375ca8c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:53.952164 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19414afd-0437-4272-a22b-bf82375ca8c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:53.952534 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19414afd-0437-4272-a22b-bf82375ca8c2 admin admin] PATH_INFO: `/v3/projects/9cff077a047a45fbbe49aa0e18626e9e` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:53.953864 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19414afd-0437-4272-a22b-bf82375ca8c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cff077a047a45fbbe49aa0e18626e9e)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:53.956174 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19414afd-0437-4272-a22b-bf82375ca8c2 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:54.035942 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-19414afd-0437-4272-a22b-bf82375ca8c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cff077a047a45fbbe49aa0e18626e9e', 'request_id': 'req-19414afd-0437-4272-a22b-bf82375ca8c2'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:54.053291 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-19414afd-0437-4272-a22b-bf82375ca8c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cff077a047a45fbbe49aa0e18626e9e', 'request_id': 'req-19414afd-0437-4272-a22b-bf82375ca8c2'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:54.066532 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-19414afd-0437-4272-a22b-bf82375ca8c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cff077a047a45fbbe49aa0e18626e9e', 'request_id': 'req-19414afd-0437-4272-a22b-bf82375ca8c2'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:54.095239 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-19414afd-0437-4272-a22b-bf82375ca8c2 admin admin] The token cache is being invalidate because project 9cff077a047a45fbbe49aa0e18626e9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:47:54.095927 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-19414afd-0437-4272-a22b-bf82375ca8c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:54.097263 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-19414afd-0437-4272-a22b-bf82375ca8c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:47:54.101026 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1239/2505] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:47:53 2022] DELETE /identity/v3/projects/9cff077a047a45fbbe49aa0e18626e9e => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:47:54.110086 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c14b9-8a79-4553-9a8e-47dc67bec276 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:54.110410 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c14b9-8a79-4553-9a8e-47dc67bec276 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:54.111051 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c14b9-8a79-4553-9a8e-47dc67bec276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:54.137908 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-444c14b9-8a79-4553-9a8e-47dc67bec276 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:47:54.139399 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-444c14b9-8a79-4553-9a8e-47dc67bec276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:54.226394 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1267/2506] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:47:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:47:54.236401 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d34998-4fcd-4377-a769-be6ee540095a None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:54.236764 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d34998-4fcd-4377-a769-be6ee540095a None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:54.237051 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d34998-4fcd-4377-a769-be6ee540095a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:54.277421 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-d4d34998-4fcd-4377-a769-be6ee540095a None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:47:54.277421 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d4d34998-4fcd-4377-a769-be6ee540095a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:54.278535 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1240/2507] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:47:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:47:54.283761 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444c14b9-8a79-4553-9a8e-47dc67bec276 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:54.295381 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444c14b9-8a79-4553-9a8e-47dc67bec276 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:54.300532 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae29ea6e-b0ee-4a53-b3c4-d099525582e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:54.301307 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29ea6e-b0ee-4a53-b3c4-d099525582e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:54.301663 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29ea6e-b0ee-4a53-b3c4-d099525582e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:54.302060 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29ea6e-b0ee-4a53-b3c4-d099525582e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:54.302888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae29ea6e-b0ee-4a53-b3c4-d099525582e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:47:54.303250 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae29ea6e-b0ee-4a53-b3c4-d099525582e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:54.305244 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae29ea6e-b0ee-4a53-b3c4-d099525582e6 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:54.323381 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1268/2508] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:47:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:47:54.325164 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d34998-4fcd-4377-a769-be6ee540095a None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:54.342126 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d34998-4fcd-4377-a769-be6ee540095a None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:54.354163 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d825a6-85b1-4333-9e13-794302d466ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:54.354163 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d825a6-85b1-4333-9e13-794302d466ee admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:54.354163 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d825a6-85b1-4333-9e13-794302d466ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:54.354163 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d825a6-85b1-4333-9e13-794302d466ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:54.354788 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d825a6-85b1-4333-9e13-794302d466ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:54.358777 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d825a6-85b1-4333-9e13-794302d466ee admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:54.406789 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1241/2509] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:47:54 2022] POST /identity/v3/projects => generated 385 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:47:54.413267 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae29ea6e-b0ee-4a53-b3c4-d099525582e6 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:54.426605 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae29ea6e-b0ee-4a53-b3c4-d099525582e6 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:54.509748 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf113fd-56c8-420f-9426-5f41bc4b0ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:54.511057 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf113fd-56c8-420f-9426-5f41bc4b0ab6 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:54.511537 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf113fd-56c8-420f-9426-5f41bc4b0ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:54.511973 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf113fd-56c8-420f-9426-5f41bc4b0ab6 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:54.513844 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf113fd-56c8-420f-9426-5f41bc4b0ab6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:54.517302 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf113fd-56c8-420f-9426-5f41bc4b0ab6 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:54.555496 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1269/2510] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:47:54 2022] POST /identity/v3/users => generated 293 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:47:54.560074 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d825a6-85b1-4333-9e13-794302d466ee admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:54.571721 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d825a6-85b1-4333-9e13-794302d466ee admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:54.577117 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fab9de-da45-41ff-9800-ebb0efb7bbf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:54.578072 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fab9de-da45-41ff-9800-ebb0efb7bbf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:54.578537 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fab9de-da45-41ff-9800-ebb0efb7bbf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:54.578827 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fab9de-da45-41ff-9800-ebb0efb7bbf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:54.579673 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fab9de-da45-41ff-9800-ebb0efb7bbf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:54.581513 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fab9de-da45-41ff-9800-ebb0efb7bbf2 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:54.597635 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1242/2511] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:47:54 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:47:54.602332 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf113fd-56c8-420f-9426-5f41bc4b0ab6 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:54.620645 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf113fd-56c8-420f-9426-5f41bc4b0ab6 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:54.629676 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda0802b-0568-4073-93dd-456c8af97769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:54.630833 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0802b-0568-4073-93dd-456c8af97769 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:54.631281 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0802b-0568-4073-93dd-456c8af97769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:54.631719 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0802b-0568-4073-93dd-456c8af97769 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:54.632998 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda0802b-0568-4073-93dd-456c8af97769 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:54.635931 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda0802b-0568-4073-93dd-456c8af97769 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:54.651695 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1270/2512] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:47:54 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:47:54.656945 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fab9de-da45-41ff-9800-ebb0efb7bbf2 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:54.668933 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fab9de-da45-41ff-9800-ebb0efb7bbf2 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:54.675466 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4b6a25-90d3-408d-9419-b9a4aebface3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:54.677152 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b6a25-90d3-408d-9419-b9a4aebface3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:54.677152 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b6a25-90d3-408d-9419-b9a4aebface3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:54.677152 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b6a25-90d3-408d-9419-b9a4aebface3 admin admin] PATH_INFO: `/v3/projects/9dc7a9c491bd42d19d790d302681437c/users/bca4f2cf3d8c4776942035c6a38425f4/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:54.699081 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4b6a25-90d3-408d-9419-b9a4aebface3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dc7a9c491bd42d19d790d302681437c, user_id=bca4f2cf3d8c4776942035c6a38425f4, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:54.702212 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4b6a25-90d3-408d-9419-b9a4aebface3 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:54.721722 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1243/2513] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:47:54 2022] PUT /identity/v3/projects/9dc7a9c491bd42d19d790d302681437c/users/bca4f2cf3d8c4776942035c6a38425f4/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:47:54.735383 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27cdd7a-8ee0-4a70-8543-4fc4e68afcb0 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:54.735943 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27cdd7a-8ee0-4a70-8543-4fc4e68afcb0 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:54.736419 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27cdd7a-8ee0-4a70-8543-4fc4e68afcb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:54.782557 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-b27cdd7a-8ee0-4a70-8543-4fc4e68afcb0 None None] MFA Rules not processed for user `bca4f2cf3d8c4776942035c6a38425f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:47:54.785388 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b27cdd7a-8ee0-4a70-8543-4fc4e68afcb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:54.868564 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:47:54.884494 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1271/2514] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:47:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:47:54.886832 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4b6a25-90d3-408d-9419-b9a4aebface3 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:54.999876 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:47:55.045780 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4b6a25-90d3-408d-9419-b9a4aebface3 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:55.061420 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea82b3-c3d3-41ec-89ef-481af43e0a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:55.062153 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea82b3-c3d3-41ec-89ef-481af43e0a5b admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:55.062445 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea82b3-c3d3-41ec-89ef-481af43e0a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:55.063362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea82b3-c3d3-41ec-89ef-481af43e0a5b admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:55.063559 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea82b3-c3d3-41ec-89ef-481af43e0a5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:55.066241 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea82b3-c3d3-41ec-89ef-481af43e0a5b admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:55.106775 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1244/2515] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:47:54 2022] POST /identity/v3/projects => generated 383 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:47:55.109002 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27cdd7a-8ee0-4a70-8543-4fc4e68afcb0 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:55.126479 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27cdd7a-8ee0-4a70-8543-4fc4e68afcb0 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:55.209562 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17615db0-4cbd-4e77-a97e-8a94d78ff392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:55.210393 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17615db0-4cbd-4e77-a97e-8a94d78ff392 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:55.210754 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17615db0-4cbd-4e77-a97e-8a94d78ff392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:55.211034 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17615db0-4cbd-4e77-a97e-8a94d78ff392 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:55.211759 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17615db0-4cbd-4e77-a97e-8a94d78ff392 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:55.213747 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17615db0-4cbd-4e77-a97e-8a94d78ff392 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:55.247346 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1272/2516] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:47:55 2022] POST /identity/v3/users => generated 292 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:47:55.252612 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea82b3-c3d3-41ec-89ef-481af43e0a5b admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:55.271410 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea82b3-c3d3-41ec-89ef-481af43e0a5b admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:55.300653 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154c0b8e-e498-4878-b1f5-ae6791e043a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:55.302809 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c0b8e-e498-4878-b1f5-ae6791e043a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:55.303067 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c0b8e-e498-4878-b1f5-ae6791e043a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:55.303203 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c0b8e-e498-4878-b1f5-ae6791e043a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:55.304371 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154c0b8e-e498-4878-b1f5-ae6791e043a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:55.306415 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154c0b8e-e498-4878-b1f5-ae6791e043a4 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:55.328062 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1245/2517] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:47:55 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:47:55.330775 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17615db0-4cbd-4e77-a97e-8a94d78ff392 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:55.390624 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17615db0-4cbd-4e77-a97e-8a94d78ff392 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:55.396335 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364acd2f-c1bf-43e7-9688-8a13178140bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:55.397048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364acd2f-c1bf-43e7-9688-8a13178140bd admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:55.397329 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364acd2f-c1bf-43e7-9688-8a13178140bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:55.397627 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364acd2f-c1bf-43e7-9688-8a13178140bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:55.399316 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364acd2f-c1bf-43e7-9688-8a13178140bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:55.401330 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364acd2f-c1bf-43e7-9688-8a13178140bd admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:55.420821 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1273/2518] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:47:55 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:47:55.422144 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154c0b8e-e498-4878-b1f5-ae6791e043a4 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:55.435010 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154c0b8e-e498-4878-b1f5-ae6791e043a4 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:55.441224 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ceef6b-b839-42dc-983a-9b1c2d9a8e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:55.442356 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ceef6b-b839-42dc-983a-9b1c2d9a8e6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:55.442742 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ceef6b-b839-42dc-983a-9b1c2d9a8e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:55.443116 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ceef6b-b839-42dc-983a-9b1c2d9a8e6a admin admin] PATH_INFO: `/v3/projects/fdcb6f15db3e4589a11c8e738046d498/users/71abc794ba364c87af2deca593239e9a/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:55.464813 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ceef6b-b839-42dc-983a-9b1c2d9a8e6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdcb6f15db3e4589a11c8e738046d498, user_id=71abc794ba364c87af2deca593239e9a, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:55.469284 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ceef6b-b839-42dc-983a-9b1c2d9a8e6a admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:55.493867 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1246/2519] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:47:55 2022] PUT /identity/v3/projects/fdcb6f15db3e4589a11c8e738046d498/users/71abc794ba364c87af2deca593239e9a/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:47:55.512031 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3be300-1b36-4c39-83fe-0c8139cc769d None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:55.512949 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3be300-1b36-4c39-83fe-0c8139cc769d None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:55.513375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3be300-1b36-4c39-83fe-0c8139cc769d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:55.559843 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-2b3be300-1b36-4c39-83fe-0c8139cc769d None None] MFA Rules not processed for user `71abc794ba364c87af2deca593239e9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:47:55.562226 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-2b3be300-1b36-4c39-83fe-0c8139cc769d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:55.697347 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1274/2520] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:47:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:47:55.703334 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ceef6b-b839-42dc-983a-9b1c2d9a8e6a admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:55.800277 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ceef6b-b839-42dc-983a-9b1c2d9a8e6a admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:55.808466 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea0367a-a11c-41e7-b619-a285e4847534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:55.809761 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea0367a-a11c-41e7-b619-a285e4847534 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:55.810266 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea0367a-a11c-41e7-b619-a285e4847534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:55.810733 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea0367a-a11c-41e7-b619-a285e4847534 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:55.812048 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea0367a-a11c-41e7-b619-a285e4847534 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:55.815835 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea0367a-a11c-41e7-b619-a285e4847534 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:55.859113 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1247/2521] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:47:55 2022] POST /identity/v3/projects => generated 385 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:47:55.863403 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3be300-1b36-4c39-83fe-0c8139cc769d None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:55.875238 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3be300-1b36-4c39-83fe-0c8139cc769d None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:55.942782 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c03ab2f-7f34-49ab-879a-4c5be4c1347c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:55.943463 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c03ab2f-7f34-49ab-879a-4c5be4c1347c admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:55.943783 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c03ab2f-7f34-49ab-879a-4c5be4c1347c admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:55.944043 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c03ab2f-7f34-49ab-879a-4c5be4c1347c admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:55.944733 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c03ab2f-7f34-49ab-879a-4c5be4c1347c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:55.948412 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c03ab2f-7f34-49ab-879a-4c5be4c1347c admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:55.959759 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:47:55.975096 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1275/2522] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:47:55 2022] POST /identity/v3/users => generated 299 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:47:55.980149 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea0367a-a11c-41e7-b619-a285e4847534 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:56.001105 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea0367a-a11c-41e7-b619-a285e4847534 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:56.009278 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4492c85-32c9-4f3e-b9b9-ae519fc658f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:56.010505 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4492c85-32c9-4f3e-b9b9-ae519fc658f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:56.011018 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4492c85-32c9-4f3e-b9b9-ae519fc658f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:56.011510 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4492c85-32c9-4f3e-b9b9-ae519fc658f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:56.012941 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4492c85-32c9-4f3e-b9b9-ae519fc658f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:56.016139 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4492c85-32c9-4f3e-b9b9-ae519fc658f5 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:56.039367 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1248/2523] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:47:55 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:47:56.045277 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c03ab2f-7f34-49ab-879a-4c5be4c1347c admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:56.065815 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c03ab2f-7f34-49ab-879a-4c5be4c1347c admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:56.074694 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ac390-81be-4017-8d05-0cdc2c5b5b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:56.076261 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ac390-81be-4017-8d05-0cdc2c5b5b6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:56.076770 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ac390-81be-4017-8d05-0cdc2c5b5b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:56.077269 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ac390-81be-4017-8d05-0cdc2c5b5b6b admin admin] PATH_INFO: `/v3/projects/b3b2d4329ca346278fe32dc14b33acfb/users/c615baed0d1d4585a07bb5a9b484a6cc/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:56.108190 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311ac390-81be-4017-8d05-0cdc2c5b5b6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3b2d4329ca346278fe32dc14b33acfb, user_id=c615baed0d1d4585a07bb5a9b484a6cc, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:56.114198 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311ac390-81be-4017-8d05-0cdc2c5b5b6b admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:56.134926 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1276/2524] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:47:56 2022] PUT /identity/v3/projects/b3b2d4329ca346278fe32dc14b33acfb/users/c615baed0d1d4585a07bb5a9b484a6cc/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:47:56.140017 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4492c85-32c9-4f3e-b9b9-ae519fc658f5 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:56.152310 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4492c85-32c9-4f3e-b9b9-ae519fc658f5 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:56.223912 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b50e29-b68d-4929-8a01-9dc668cd30ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:56.224919 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b50e29-b68d-4929-8a01-9dc668cd30ae admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:56.225408 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b50e29-b68d-4929-8a01-9dc668cd30ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:56.225941 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b50e29-b68d-4929-8a01-9dc668cd30ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:56.227286 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b50e29-b68d-4929-8a01-9dc668cd30ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:56.230541 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b50e29-b68d-4929-8a01-9dc668cd30ae admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:56.251161 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1249/2525] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:47:56 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:47:56.256090 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ac390-81be-4017-8d05-0cdc2c5b5b6b admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:56.267736 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ac390-81be-4017-8d05-0cdc2c5b5b6b admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:56.274030 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddddb3f-3e40-46c2-8167-3db76d32ed6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:56.275058 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddddb3f-3e40-46c2-8167-3db76d32ed6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:56.275388 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddddb3f-3e40-46c2-8167-3db76d32ed6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:56.275705 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddddb3f-3e40-46c2-8167-3db76d32ed6f admin admin] PATH_INFO: `/v3/projects/b3b2d4329ca346278fe32dc14b33acfb/users/c615baed0d1d4585a07bb5a9b484a6cc/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:56.281045 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddddb3f-3e40-46c2-8167-3db76d32ed6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3b2d4329ca346278fe32dc14b33acfb, user_id=c615baed0d1d4585a07bb5a9b484a6cc, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:56.284461 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddddb3f-3e40-46c2-8167-3db76d32ed6f admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:56.299024 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1277/2526] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:47:56 2022] PUT /identity/v3/projects/b3b2d4329ca346278fe32dc14b33acfb/users/c615baed0d1d4585a07bb5a9b484a6cc/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:47:56.312443 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d2386-1d50-4fe3-8580-1108a446d236 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:56.312827 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d2386-1d50-4fe3-8580-1108a446d236 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:56.313212 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d2386-1d50-4fe3-8580-1108a446d236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:56.320684 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:47:56.347396 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-357d2386-1d50-4fe3-8580-1108a446d236 None None] MFA Rules not processed for user `c615baed0d1d4585a07bb5a9b484a6cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:47:56.348969 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-357d2386-1d50-4fe3-8580-1108a446d236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:56.435445 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1250/2527] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:47:56 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:47:56.447573 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddddb3f-3e40-46c2-8167-3db76d32ed6f admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:56.450368 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-5ddddb3f-3e40-46c2-8167-3db76d32ed6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:56.528655 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddddb3f-3e40-46c2-8167-3db76d32ed6f admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:56.532440 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1907dff-5f6e-4cfb-96c3-c139ca31b278 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:56.533104 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1907dff-5f6e-4cfb-96c3-c139ca31b278 service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:56.533387 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1907dff-5f6e-4cfb-96c3-c139ca31b278 service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:56.533664 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1907dff-5f6e-4cfb-96c3-c139ca31b278 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:56.608956 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1907dff-5f6e-4cfb-96c3-c139ca31b278 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:56.610724 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1907dff-5f6e-4cfb-96c3-c139ca31b278 service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:56.622676 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1278/2528] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:47:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:47:57.475197 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18beecdb-446d-4c08-b6cf-faaf94f41fdb None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:57.475985 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18beecdb-446d-4c08-b6cf-faaf94f41fdb None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:57.476182 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18beecdb-446d-4c08-b6cf-faaf94f41fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:57.501413 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-18beecdb-446d-4c08-b6cf-faaf94f41fdb None None] MFA Rules not processed for user `85d9c4c57681447c898ebfd7990b05b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:47:57.503067 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-18beecdb-446d-4c08-b6cf-faaf94f41fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:57.581319 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1251/2529] 10.222.0.13 () {58 vars in 967 bytes} [Fri Jun 24 08:47:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:47:57.594429 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1907dff-5f6e-4cfb-96c3-c139ca31b278 service cinder] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:57.595964 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-f1907dff-5f6e-4cfb-96c3-c139ca31b278 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:57.674732 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1907dff-5f6e-4cfb-96c3-c139ca31b278 service cinder] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:57.680149 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccde499-479b-4a79-b745-3376ebba23f0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec8c2f113cf41a0939b81f38fbee813', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:57.680922 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccde499-479b-4a79-b745-3376ebba23f0 service swift] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:57.681252 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccde499-479b-4a79-b745-3376ebba23f0 service swift] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:57.681672 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccde499-479b-4a79-b745-3376ebba23f0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:57.692529 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccde499-479b-4a79-b745-3376ebba23f0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:57.694247 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccde499-479b-4a79-b745-3376ebba23f0 service swift] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:57.699856 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:47:57.706792 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1279/2530] 10.222.0.13 () {60 vars in 1384 bytes} [Fri Jun 24 08:47:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 08:47:58.279099 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18beecdb-446d-4c08-b6cf-faaf94f41fdb None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:58.281908 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-18beecdb-446d-4c08-b6cf-faaf94f41fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:58.297853 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18beecdb-446d-4c08-b6cf-faaf94f41fdb None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:58.301723 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d3c8e5-06aa-46a6-ad44-08abb9549cd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:58.302394 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d3c8e5-06aa-46a6-ad44-08abb9549cd5 service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:58.302684 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d3c8e5-06aa-46a6-ad44-08abb9549cd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:58.302968 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d3c8e5-06aa-46a6-ad44-08abb9549cd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:58.395555 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d3c8e5-06aa-46a6-ad44-08abb9549cd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:58.398429 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d3c8e5-06aa-46a6-ad44-08abb9549cd5 service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:58.416891 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1252/2531] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:47:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:47:59.124485 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccde499-479b-4a79-b745-3376ebba23f0 service swift] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:59.140264 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccde499-479b-4a79-b745-3376ebba23f0 service swift] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:59.146237 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb633dec-5ede-4d90-a666-da86780c6bb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:59.147362 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb633dec-5ede-4d90-a666-da86780c6bb9 service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:59.147861 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb633dec-5ede-4d90-a666-da86780c6bb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:59.148351 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb633dec-5ede-4d90-a666-da86780c6bb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:59.164858 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb633dec-5ede-4d90-a666-da86780c6bb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:59.167627 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb633dec-5ede-4d90-a666-da86780c6bb9 service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:59.186005 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1280/2532] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:47:59 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:48:03.175026 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d3c8e5-06aa-46a6-ad44-08abb9549cd5 service cinder] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:03.214846 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:48:03.248370 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d3c8e5-06aa-46a6-ad44-08abb9549cd5 service cinder] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:03.253506 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3f802-fe35-45d8-a839-a9dee6548cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:03.254413 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3f802-fe35-45d8-a839-a9dee6548cbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:03.254698 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3f802-fe35-45d8-a839-a9dee6548cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:03.254973 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3f802-fe35-45d8-a839-a9dee6548cbf admin admin] PATH_INFO: `/v3/users/bca4f2cf3d8c4776942035c6a38425f4` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:03.256236 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc3f802-fe35-45d8-a839-a9dee6548cbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=bca4f2cf3d8c4776942035c6a38425f4)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:03.258235 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc3f802-fe35-45d8-a839-a9dee6548cbf admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:03.308977 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-5fc3f802-fe35-45d8-a839-a9dee6548cbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bca4f2cf3d8c4776942035c6a38425f4'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:03.321444 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-5fc3f802-fe35-45d8-a839-a9dee6548cbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bca4f2cf3d8c4776942035c6a38425f4'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:03.322246 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-5fc3f802-fe35-45d8-a839-a9dee6548cbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bca4f2cf3d8c4776942035c6a38425f4'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:03.330177 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-5fc3f802-fe35-45d8-a839-a9dee6548cbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bca4f2cf3d8c4776942035c6a38425f4'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:03.330899 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-5fc3f802-fe35-45d8-a839-a9dee6548cbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bca4f2cf3d8c4776942035c6a38425f4'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:03.346349 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1253/2533] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:48:03 2022] DELETE /identity/v3/users/bca4f2cf3d8c4776942035c6a38425f4 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:48:03.351002 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5f7a2-4d41-449f-a842-a1e6132c9b7f None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:03.351875 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5f7a2-4d41-449f-a842-a1e6132c9b7f None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:03.352139 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5f7a2-4d41-449f-a842-a1e6132c9b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:03.371022 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-2ae5f7a2-4d41-449f-a842-a1e6132c9b7f None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:48:03.372476 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-2ae5f7a2-4d41-449f-a842-a1e6132c9b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:03.440864 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1281/2534] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:48:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:48:03.451684 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3f802-fe35-45d8-a839-a9dee6548cbf admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:03.453146 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-5fc3f802-fe35-45d8-a839-a9dee6548cbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:03.526160 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3f802-fe35-45d8-a839-a9dee6548cbf admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:03.530302 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a867d3e-891f-4c0c-b1e5-21fb9a4f2c59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:03.530955 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a867d3e-891f-4c0c-b1e5-21fb9a4f2c59 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:03.531238 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a867d3e-891f-4c0c-b1e5-21fb9a4f2c59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:03.531513 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a867d3e-891f-4c0c-b1e5-21fb9a4f2c59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:03.541792 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a867d3e-891f-4c0c-b1e5-21fb9a4f2c59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:03.543818 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a867d3e-891f-4c0c-b1e5-21fb9a4f2c59 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:03.556499 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1254/2535] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:48:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:48:03.897462 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae5f7a2-4d41-449f-a842-a1e6132c9b7f None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:03.899397 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-2ae5f7a2-4d41-449f-a842-a1e6132c9b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:03.915261 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae5f7a2-4d41-449f-a842-a1e6132c9b7f None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:03.923599 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7aacd5-32e1-4259-be30-f4d9fa1f7aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:03.924397 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7aacd5-32e1-4259-be30-f4d9fa1f7aca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:03.924678 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7aacd5-32e1-4259-be30-f4d9fa1f7aca admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:03.924947 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7aacd5-32e1-4259-be30-f4d9fa1f7aca admin admin] PATH_INFO: `/v3/projects/9dc7a9c491bd42d19d790d302681437c` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:03.926531 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7aacd5-32e1-4259-be30-f4d9fa1f7aca admin admin] RBAC: Authorizing `identity:delete_project(project_id=9dc7a9c491bd42d19d790d302681437c)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:03.928493 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7aacd5-32e1-4259-be30-f4d9fa1f7aca admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:03.991833 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7e7aacd5-32e1-4259-be30-f4d9fa1f7aca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9dc7a9c491bd42d19d790d302681437c', 'request_id': 'req-7e7aacd5-32e1-4259-be30-f4d9fa1f7aca'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:04.000197 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7e7aacd5-32e1-4259-be30-f4d9fa1f7aca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9dc7a9c491bd42d19d790d302681437c', 'request_id': 'req-7e7aacd5-32e1-4259-be30-f4d9fa1f7aca'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:04.007397 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7e7aacd5-32e1-4259-be30-f4d9fa1f7aca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9dc7a9c491bd42d19d790d302681437c', 'request_id': 'req-7e7aacd5-32e1-4259-be30-f4d9fa1f7aca'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:04.023181 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7e7aacd5-32e1-4259-be30-f4d9fa1f7aca admin admin] The token cache is being invalidate because project 9dc7a9c491bd42d19d790d302681437c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:48:04.023516 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7e7aacd5-32e1-4259-be30-f4d9fa1f7aca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:04.024125 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7e7aacd5-32e1-4259-be30-f4d9fa1f7aca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:04.025789 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1282/2536] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:48:03 2022] DELETE /identity/v3/projects/9dc7a9c491bd42d19d790d302681437c => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:48:04.030556 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a867d3e-891f-4c0c-b1e5-21fb9a4f2c59 service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:04.041271 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a867d3e-891f-4c0c-b1e5-21fb9a4f2c59 service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:04.102283 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:48:04.115049 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cdd1ea-aeeb-4a31-b07d-6fce17a5e4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:04.115960 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cdd1ea-aeeb-4a31-b07d-6fce17a5e4fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:04.116301 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cdd1ea-aeeb-4a31-b07d-6fce17a5e4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:04.116949 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cdd1ea-aeeb-4a31-b07d-6fce17a5e4fe admin admin] PATH_INFO: `/v3/users/71abc794ba364c87af2deca593239e9a` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:04.118740 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cdd1ea-aeeb-4a31-b07d-6fce17a5e4fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=71abc794ba364c87af2deca593239e9a)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:04.120970 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cdd1ea-aeeb-4a31-b07d-6fce17a5e4fe admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:04.178263 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f9cdd1ea-aeeb-4a31-b07d-6fce17a5e4fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71abc794ba364c87af2deca593239e9a'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:04.204077 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f9cdd1ea-aeeb-4a31-b07d-6fce17a5e4fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71abc794ba364c87af2deca593239e9a'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:04.205251 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f9cdd1ea-aeeb-4a31-b07d-6fce17a5e4fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71abc794ba364c87af2deca593239e9a'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:04.221791 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f9cdd1ea-aeeb-4a31-b07d-6fce17a5e4fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71abc794ba364c87af2deca593239e9a'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:04.222982 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f9cdd1ea-aeeb-4a31-b07d-6fce17a5e4fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71abc794ba364c87af2deca593239e9a'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:04.244327 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1255/2537] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:48:04 2022] DELETE /identity/v3/users/71abc794ba364c87af2deca593239e9a => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:48:04.640360 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7aacd5-32e1-4259-be30-f4d9fa1f7aca admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:04.642295 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-7e7aacd5-32e1-4259-be30-f4d9fa1f7aca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:04.661950 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7aacd5-32e1-4259-be30-f4d9fa1f7aca admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:04.746723 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87a7432-0a9c-4b13-917a-1b78820dee1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:04.747607 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87a7432-0a9c-4b13-917a-1b78820dee1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:04.747889 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87a7432-0a9c-4b13-917a-1b78820dee1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:04.748166 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87a7432-0a9c-4b13-917a-1b78820dee1b admin admin] PATH_INFO: `/v3/projects/fdcb6f15db3e4589a11c8e738046d498` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:04.749443 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87a7432-0a9c-4b13-917a-1b78820dee1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdcb6f15db3e4589a11c8e738046d498)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:04.751426 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87a7432-0a9c-4b13-917a-1b78820dee1b admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:04.764789 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:48:04.809681 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d87a7432-0a9c-4b13-917a-1b78820dee1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdcb6f15db3e4589a11c8e738046d498', 'request_id': 'req-d87a7432-0a9c-4b13-917a-1b78820dee1b'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:04.817114 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d87a7432-0a9c-4b13-917a-1b78820dee1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdcb6f15db3e4589a11c8e738046d498', 'request_id': 'req-d87a7432-0a9c-4b13-917a-1b78820dee1b'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:04.825288 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d87a7432-0a9c-4b13-917a-1b78820dee1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdcb6f15db3e4589a11c8e738046d498', 'request_id': 'req-d87a7432-0a9c-4b13-917a-1b78820dee1b'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:04.841622 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d87a7432-0a9c-4b13-917a-1b78820dee1b admin admin] The token cache is being invalidate because project fdcb6f15db3e4589a11c8e738046d498 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:48:04.841974 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d87a7432-0a9c-4b13-917a-1b78820dee1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:04.842867 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d87a7432-0a9c-4b13-917a-1b78820dee1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:04.844994 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1283/2538] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:48:04 2022] DELETE /identity/v3/projects/fdcb6f15db3e4589a11c8e738046d498 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:48:04.849669 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cdd1ea-aeeb-4a31-b07d-6fce17a5e4fe admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:04.851306 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-f9cdd1ea-aeeb-4a31-b07d-6fce17a5e4fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:04.866668 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cdd1ea-aeeb-4a31-b07d-6fce17a5e4fe admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:04.941056 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a10ce-e861-4096-a41f-dab67cabcca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:04.943082 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a10ce-e861-4096-a41f-dab67cabcca5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:04.943694 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a10ce-e861-4096-a41f-dab67cabcca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:04.944160 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a10ce-e861-4096-a41f-dab67cabcca5 admin admin] PATH_INFO: `/v3/users/c615baed0d1d4585a07bb5a9b484a6cc` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:04.945918 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715a10ce-e861-4096-a41f-dab67cabcca5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c615baed0d1d4585a07bb5a9b484a6cc)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:04.947939 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715a10ce-e861-4096-a41f-dab67cabcca5 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:05.012183 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-715a10ce-e861-4096-a41f-dab67cabcca5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c615baed0d1d4585a07bb5a9b484a6cc'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:05.026641 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-715a10ce-e861-4096-a41f-dab67cabcca5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c615baed0d1d4585a07bb5a9b484a6cc'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:05.027457 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-715a10ce-e861-4096-a41f-dab67cabcca5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c615baed0d1d4585a07bb5a9b484a6cc'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:05.037650 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-715a10ce-e861-4096-a41f-dab67cabcca5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c615baed0d1d4585a07bb5a9b484a6cc'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:05.038529 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-715a10ce-e861-4096-a41f-dab67cabcca5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c615baed0d1d4585a07bb5a9b484a6cc'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:05.041681 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:48:05.060457 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1256/2539] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:48:04 2022] DELETE /identity/v3/users/c615baed0d1d4585a07bb5a9b484a6cc => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:48:05.413187 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87a7432-0a9c-4b13-917a-1b78820dee1b admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:05.415488 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-d87a7432-0a9c-4b13-917a-1b78820dee1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:05.437947 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87a7432-0a9c-4b13-917a-1b78820dee1b admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:05.520060 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa780f8-49ec-413e-b808-3cea056c6504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:05.520949 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa780f8-49ec-413e-b808-3cea056c6504 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:05.521230 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa780f8-49ec-413e-b808-3cea056c6504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:05.521506 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa780f8-49ec-413e-b808-3cea056c6504 admin admin] PATH_INFO: `/v3/projects/b3b2d4329ca346278fe32dc14b33acfb` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:05.522819 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa780f8-49ec-413e-b808-3cea056c6504 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3b2d4329ca346278fe32dc14b33acfb)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:05.524755 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa780f8-49ec-413e-b808-3cea056c6504 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:05.583310 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-afa780f8-49ec-413e-b808-3cea056c6504 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3b2d4329ca346278fe32dc14b33acfb', 'request_id': 'req-afa780f8-49ec-413e-b808-3cea056c6504'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:05.591233 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-afa780f8-49ec-413e-b808-3cea056c6504 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3b2d4329ca346278fe32dc14b33acfb', 'request_id': 'req-afa780f8-49ec-413e-b808-3cea056c6504'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:05.597975 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-afa780f8-49ec-413e-b808-3cea056c6504 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3b2d4329ca346278fe32dc14b33acfb', 'request_id': 'req-afa780f8-49ec-413e-b808-3cea056c6504'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:05.614390 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-afa780f8-49ec-413e-b808-3cea056c6504 admin admin] The token cache is being invalidate because project b3b2d4329ca346278fe32dc14b33acfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:48:05.614688 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-afa780f8-49ec-413e-b808-3cea056c6504 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:05.615353 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-afa780f8-49ec-413e-b808-3cea056c6504 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:05.617005 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1284/2540] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:48:05 2022] DELETE /identity/v3/projects/b3b2d4329ca346278fe32dc14b33acfb => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:48:05.623301 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c79d29-fc99-4333-80ac-afa7f0be2fad None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:05.623904 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c79d29-fc99-4333-80ac-afa7f0be2fad None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:05.623904 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c79d29-fc99-4333-80ac-afa7f0be2fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:05.647461 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-e0c79d29-fc99-4333-80ac-afa7f0be2fad None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:48:05.648733 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-e0c79d29-fc99-4333-80ac-afa7f0be2fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:05.727634 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1257/2541] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:48:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:48:05.735988 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d2f513-7af7-450f-984e-29f08dd8ee7f None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:05.736575 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d2f513-7af7-450f-984e-29f08dd8ee7f None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:05.736575 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d2f513-7af7-450f-984e-29f08dd8ee7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:05.756988 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-b7d2f513-7af7-450f-984e-29f08dd8ee7f None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:48:05.758576 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b7d2f513-7af7-450f-984e-29f08dd8ee7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:05.767303 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1285/2542] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:48:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:48:05.771832 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c79d29-fc99-4333-80ac-afa7f0be2fad None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:05.784062 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c79d29-fc99-4333-80ac-afa7f0be2fad None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:05.789293 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd12a5b-171d-4f44-a0c4-5e9e25bef189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:05.790064 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd12a5b-171d-4f44-a0c4-5e9e25bef189 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:05.790387 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd12a5b-171d-4f44-a0c4-5e9e25bef189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:05.790703 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd12a5b-171d-4f44-a0c4-5e9e25bef189 admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:05.791478 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd12a5b-171d-4f44-a0c4-5e9e25bef189 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:48:05.791824 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd12a5b-171d-4f44-a0c4-5e9e25bef189 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:05.793587 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd12a5b-171d-4f44-a0c4-5e9e25bef189 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:05.806883 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1258/2543] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:48:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:48:05.812293 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d2f513-7af7-450f-984e-29f08dd8ee7f None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:05.822227 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d2f513-7af7-450f-984e-29f08dd8ee7f None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:05.827505 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a531cab3-74ae-44ac-b4d2-b74f16d6e569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:05.828249 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a531cab3-74ae-44ac-b4d2-b74f16d6e569 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:05.828510 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a531cab3-74ae-44ac-b4d2-b74f16d6e569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:05.828792 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a531cab3-74ae-44ac-b4d2-b74f16d6e569 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:05.829511 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a531cab3-74ae-44ac-b4d2-b74f16d6e569 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:05.831491 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a531cab3-74ae-44ac-b4d2-b74f16d6e569 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:05.861334 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1286/2544] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:48:05 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:48:05.865795 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd12a5b-171d-4f44-a0c4-5e9e25bef189 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:05.877930 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd12a5b-171d-4f44-a0c4-5e9e25bef189 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:05.941895 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645a94f-e808-4276-922d-e8f9e374b7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:05.942575 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645a94f-e808-4276-922d-e8f9e374b7f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:05.942856 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645a94f-e808-4276-922d-e8f9e374b7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:05.943129 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645a94f-e808-4276-922d-e8f9e374b7f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:05.943880 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6645a94f-e808-4276-922d-e8f9e374b7f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:05.946090 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6645a94f-e808-4276-922d-e8f9e374b7f7 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:05.969222 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1259/2545] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:48:05 2022] POST /identity/v3/users => generated 286 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:48:05.974233 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a531cab3-74ae-44ac-b4d2-b74f16d6e569 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:05.983942 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:48:05.994682 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a531cab3-74ae-44ac-b4d2-b74f16d6e569 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:06.001995 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457d018c-3889-4fdf-8eb7-e8de634e6e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:06.002982 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d018c-3889-4fdf-8eb7-e8de634e6e7a admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:06.003298 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d018c-3889-4fdf-8eb7-e8de634e6e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:06.003549 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d018c-3889-4fdf-8eb7-e8de634e6e7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:06.004293 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457d018c-3889-4fdf-8eb7-e8de634e6e7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:06.005959 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457d018c-3889-4fdf-8eb7-e8de634e6e7a admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:06.019342 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1287/2546] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:48:05 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:48:06.025128 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645a94f-e808-4276-922d-e8f9e374b7f7 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:06.037640 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645a94f-e808-4276-922d-e8f9e374b7f7 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:06.042826 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b81be2-2a4f-4aa1-9319-3ebcec988e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:06.043558 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b81be2-2a4f-4aa1-9319-3ebcec988e54 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:06.043848 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b81be2-2a4f-4aa1-9319-3ebcec988e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:06.044127 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b81be2-2a4f-4aa1-9319-3ebcec988e54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:06.044867 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b81be2-2a4f-4aa1-9319-3ebcec988e54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:06.046869 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b81be2-2a4f-4aa1-9319-3ebcec988e54 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:06.060985 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1260/2547] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:48:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:48:06.066509 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457d018c-3889-4fdf-8eb7-e8de634e6e7a admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:06.084467 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457d018c-3889-4fdf-8eb7-e8de634e6e7a admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:06.092641 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274b0412-6547-44a4-b0b8-c4fa09fd0c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:06.094274 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b0412-6547-44a4-b0b8-c4fa09fd0c4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:06.094782 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b0412-6547-44a4-b0b8-c4fa09fd0c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:06.095279 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b0412-6547-44a4-b0b8-c4fa09fd0c4c admin admin] PATH_INFO: `/v3/projects/a2e37ac961f54d5997e1fdbb1c67b4f3/users/a4bd35bc694c4a2184e8bab3e4540067/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:06.122429 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274b0412-6547-44a4-b0b8-c4fa09fd0c4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2e37ac961f54d5997e1fdbb1c67b4f3, user_id=a4bd35bc694c4a2184e8bab3e4540067, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:06.128215 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274b0412-6547-44a4-b0b8-c4fa09fd0c4c admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:06.149570 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1288/2548] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:48:06 2022] PUT /identity/v3/projects/a2e37ac961f54d5997e1fdbb1c67b4f3/users/a4bd35bc694c4a2184e8bab3e4540067/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:48:06.168760 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f758660-b18a-4b5d-ba4f-e2ae16f08578 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:06.169057 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f758660-b18a-4b5d-ba4f-e2ae16f08578 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:06.169337 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f758660-b18a-4b5d-ba4f-e2ae16f08578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:06.200115 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-5f758660-b18a-4b5d-ba4f-e2ae16f08578 None None] MFA Rules not processed for user `a4bd35bc694c4a2184e8bab3e4540067`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:48:06.201373 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-5f758660-b18a-4b5d-ba4f-e2ae16f08578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:06.254044 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:48:06.276402 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1261/2549] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:48:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:48:06.288254 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274b0412-6547-44a4-b0b8-c4fa09fd0c4c admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:06.290162 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-274b0412-6547-44a4-b0b8-c4fa09fd0c4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:06.355503 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274b0412-6547-44a4-b0b8-c4fa09fd0c4c admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:06.361659 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee42c3-206a-48fb-878b-a26fbd57af98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:06.362486 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee42c3-206a-48fb-878b-a26fbd57af98 service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:06.362912 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee42c3-206a-48fb-878b-a26fbd57af98 service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:06.363315 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee42c3-206a-48fb-878b-a26fbd57af98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:06.374208 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bee42c3-206a-48fb-878b-a26fbd57af98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:06.375974 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bee42c3-206a-48fb-878b-a26fbd57af98 service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:06.387594 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1289/2550] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:48:06 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:48:06.424521 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f758660-b18a-4b5d-ba4f-e2ae16f08578 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:06.518932 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f758660-b18a-4b5d-ba4f-e2ae16f08578 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:06.526066 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8d8da2-e0b5-445f-8262-398efe210492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:06.528123 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d8da2-e0b5-445f-8262-398efe210492 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:06.528629 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d8da2-e0b5-445f-8262-398efe210492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:06.529165 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d8da2-e0b5-445f-8262-398efe210492 admin admin] PATH_INFO: `/v3/users/a4bd35bc694c4a2184e8bab3e4540067` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:06.531341 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8d8da2-e0b5-445f-8262-398efe210492 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4bd35bc694c4a2184e8bab3e4540067)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:06.535125 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8d8da2-e0b5-445f-8262-398efe210492 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:06.591953 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-0c8d8da2-e0b5-445f-8262-398efe210492 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4bd35bc694c4a2184e8bab3e4540067'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:06.605744 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-0c8d8da2-e0b5-445f-8262-398efe210492 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4bd35bc694c4a2184e8bab3e4540067'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:06.607004 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-0c8d8da2-e0b5-445f-8262-398efe210492 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4bd35bc694c4a2184e8bab3e4540067'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:06.614648 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-0c8d8da2-e0b5-445f-8262-398efe210492 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4bd35bc694c4a2184e8bab3e4540067'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:06.615412 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-0c8d8da2-e0b5-445f-8262-398efe210492 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4bd35bc694c4a2184e8bab3e4540067'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:06.631309 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1262/2551] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:48:06 2022] DELETE /identity/v3/users/a4bd35bc694c4a2184e8bab3e4540067 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:48:06.636201 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f62957-55c4-45e1-b87a-d3778d5afcda None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:06.636497 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f62957-55c4-45e1-b87a-d3778d5afcda None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:06.636786 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f62957-55c4-45e1-b87a-d3778d5afcda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:06.656951 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-81f62957-55c4-45e1-b87a-d3778d5afcda None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:48:06.658536 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-81f62957-55c4-45e1-b87a-d3778d5afcda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:06.752869 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1290/2552] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:48:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:48:06.764147 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8d8da2-e0b5-445f-8262-398efe210492 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:06.765680 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-0c8d8da2-e0b5-445f-8262-398efe210492 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:06.831415 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8d8da2-e0b5-445f-8262-398efe210492 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:06.835116 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992ae916-663c-4d8c-992d-15d3af98778c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:06.835774 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ae916-663c-4d8c-992d-15d3af98778c service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:06.836061 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ae916-663c-4d8c-992d-15d3af98778c service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:06.836457 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ae916-663c-4d8c-992d-15d3af98778c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:06.847403 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992ae916-663c-4d8c-992d-15d3af98778c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:06.848977 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992ae916-663c-4d8c-992d-15d3af98778c service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:06.860149 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1263/2553] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:48:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:48:07.258783 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f62957-55c4-45e1-b87a-d3778d5afcda None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:07.261372 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-81f62957-55c4-45e1-b87a-d3778d5afcda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:07.277371 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f62957-55c4-45e1-b87a-d3778d5afcda None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:07.283557 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efe923b-33d4-41b5-885f-a7330d885a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:07.284488 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe923b-33d4-41b5-885f-a7330d885a42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:07.284772 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe923b-33d4-41b5-885f-a7330d885a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:07.285209 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe923b-33d4-41b5-885f-a7330d885a42 admin admin] PATH_INFO: `/v3/projects/a2e37ac961f54d5997e1fdbb1c67b4f3` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:07.286540 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efe923b-33d4-41b5-885f-a7330d885a42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2e37ac961f54d5997e1fdbb1c67b4f3)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:07.288478 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efe923b-33d4-41b5-885f-a7330d885a42 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:07.340830 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-8efe923b-33d4-41b5-885f-a7330d885a42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2e37ac961f54d5997e1fdbb1c67b4f3', 'request_id': 'req-8efe923b-33d4-41b5-885f-a7330d885a42'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:07.351417 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-8efe923b-33d4-41b5-885f-a7330d885a42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2e37ac961f54d5997e1fdbb1c67b4f3', 'request_id': 'req-8efe923b-33d4-41b5-885f-a7330d885a42'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:07.360562 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-8efe923b-33d4-41b5-885f-a7330d885a42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2e37ac961f54d5997e1fdbb1c67b4f3', 'request_id': 'req-8efe923b-33d4-41b5-885f-a7330d885a42'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:07.363567 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:48:07.381460 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-8efe923b-33d4-41b5-885f-a7330d885a42 admin admin] The token cache is being invalidate because project a2e37ac961f54d5997e1fdbb1c67b4f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:48:07.381895 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-8efe923b-33d4-41b5-885f-a7330d885a42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:07.382698 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-8efe923b-33d4-41b5-885f-a7330d885a42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:07.384449 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1291/2554] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:48:07 2022] DELETE /identity/v3/projects/a2e37ac961f54d5997e1fdbb1c67b4f3 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:48:07.390043 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac38b5f-d817-4a16-8783-5f916976b4cc None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:07.390332 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac38b5f-d817-4a16-8783-5f916976b4cc None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:07.390642 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac38b5f-d817-4a16-8783-5f916976b4cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:07.410996 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-2ac38b5f-d817-4a16-8783-5f916976b4cc None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:48:07.412247 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-2ac38b5f-d817-4a16-8783-5f916976b4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:07.418921 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:48:07.504929 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1264/2555] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:48:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:48:07.512905 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a4b588-064f-4ae1-96c8-5d59aabb8143 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:07.513280 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a4b588-064f-4ae1-96c8-5d59aabb8143 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:07.513603 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a4b588-064f-4ae1-96c8-5d59aabb8143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:07.540930 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-f9a4b588-064f-4ae1-96c8-5d59aabb8143 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:48:07.543123 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-f9a4b588-064f-4ae1-96c8-5d59aabb8143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:07.557559 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1292/2556] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:48:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:48:07.562326 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac38b5f-d817-4a16-8783-5f916976b4cc None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:07.572527 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac38b5f-d817-4a16-8783-5f916976b4cc None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:07.577904 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd3907a-7b94-4bfc-9515-723b2af62153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:07.578671 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd3907a-7b94-4bfc-9515-723b2af62153 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:07.579006 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd3907a-7b94-4bfc-9515-723b2af62153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:07.579321 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd3907a-7b94-4bfc-9515-723b2af62153 admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:07.580099 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd3907a-7b94-4bfc-9515-723b2af62153 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:48:07.580445 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd3907a-7b94-4bfc-9515-723b2af62153 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:07.582175 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd3907a-7b94-4bfc-9515-723b2af62153 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:07.596516 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1265/2557] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:48:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:48:07.604014 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a4b588-064f-4ae1-96c8-5d59aabb8143 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:07.622156 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a4b588-064f-4ae1-96c8-5d59aabb8143 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:07.631733 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5477a1-8e60-4109-83b4-b1739a4b483f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:07.632653 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5477a1-8e60-4109-83b4-b1739a4b483f admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:07.632938 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5477a1-8e60-4109-83b4-b1739a4b483f admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:07.633238 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5477a1-8e60-4109-83b4-b1739a4b483f admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:07.634424 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5477a1-8e60-4109-83b4-b1739a4b483f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:07.638581 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5477a1-8e60-4109-83b4-b1739a4b483f admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:07.689139 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1293/2558] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:48:07 2022] POST /identity/v3/projects => generated 375 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:48:07.695222 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd3907a-7b94-4bfc-9515-723b2af62153 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:07.707577 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd3907a-7b94-4bfc-9515-723b2af62153 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:07.777065 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e12a55-5440-46f9-be53-53e62f6a594a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:07.777858 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e12a55-5440-46f9-be53-53e62f6a594a admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:07.778385 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e12a55-5440-46f9-be53-53e62f6a594a admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:07.778792 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e12a55-5440-46f9-be53-53e62f6a594a admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:07.779518 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e12a55-5440-46f9-be53-53e62f6a594a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:07.781490 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e12a55-5440-46f9-be53-53e62f6a594a admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:07.808355 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1266/2559] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:48:07 2022] POST /identity/v3/users => generated 288 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:48:07.812868 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5477a1-8e60-4109-83b4-b1739a4b483f admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:07.824003 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5477a1-8e60-4109-83b4-b1739a4b483f admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:07.829415 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d78ce37-da02-46f9-8e3b-58ae7908d4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:07.830138 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78ce37-da02-46f9-8e3b-58ae7908d4c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:07.830420 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78ce37-da02-46f9-8e3b-58ae7908d4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:07.830692 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78ce37-da02-46f9-8e3b-58ae7908d4c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:07.831439 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d78ce37-da02-46f9-8e3b-58ae7908d4c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:07.833054 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d78ce37-da02-46f9-8e3b-58ae7908d4c1 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:07.845065 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1294/2560] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:48:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:48:07.849535 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e12a55-5440-46f9-be53-53e62f6a594a admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:07.861982 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e12a55-5440-46f9-be53-53e62f6a594a admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:07.867362 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb59e1e-26be-475a-83b0-4f377b067cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:07.868090 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb59e1e-26be-475a-83b0-4f377b067cd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:07.868423 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb59e1e-26be-475a-83b0-4f377b067cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:07.868753 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb59e1e-26be-475a-83b0-4f377b067cd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:07.869543 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb59e1e-26be-475a-83b0-4f377b067cd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:07.871538 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb59e1e-26be-475a-83b0-4f377b067cd3 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:07.883700 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1267/2561] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:48:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:48:07.888846 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d78ce37-da02-46f9-8e3b-58ae7908d4c1 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:07.900162 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d78ce37-da02-46f9-8e3b-58ae7908d4c1 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:07.905711 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aa427d-611e-43fb-bc54-e0c261e1a3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:07.906708 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa427d-611e-43fb-bc54-e0c261e1a3f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:07.907031 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa427d-611e-43fb-bc54-e0c261e1a3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:07.907347 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa427d-611e-43fb-bc54-e0c261e1a3f8 admin admin] PATH_INFO: `/v3/projects/0d4721b2f8834f2fa81aa7d0d5f090e2/users/10e2d2ab314c4c55a4100384d93ea93f/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:07.925119 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aa427d-611e-43fb-bc54-e0c261e1a3f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d4721b2f8834f2fa81aa7d0d5f090e2, user_id=10e2d2ab314c4c55a4100384d93ea93f, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:07.927985 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aa427d-611e-43fb-bc54-e0c261e1a3f8 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:07.947388 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1295/2562] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:48:07 2022] PUT /identity/v3/projects/0d4721b2f8834f2fa81aa7d0d5f090e2/users/10e2d2ab314c4c55a4100384d93ea93f/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:48:07.966503 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7125d4-faf8-4179-a7de-4ba5d6f8ca3c None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:07.966791 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7125d4-faf8-4179-a7de-4ba5d6f8ca3c None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:07.967153 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7125d4-faf8-4179-a7de-4ba5d6f8ca3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:07.998457 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-7d7125d4-faf8-4179-a7de-4ba5d6f8ca3c None None] MFA Rules not processed for user `10e2d2ab314c4c55a4100384d93ea93f`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:48:07.999705 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-7d7125d4-faf8-4179-a7de-4ba5d6f8ca3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:08.077539 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1268/2563] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:48:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:48:08.088627 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aa427d-611e-43fb-bc54-e0c261e1a3f8 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:08.091537 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-16aa427d-611e-43fb-bc54-e0c261e1a3f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:08.166045 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aa427d-611e-43fb-bc54-e0c261e1a3f8 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:08.171976 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5956242c-c9f1-4bc4-82e8-ab9bdf249ca9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:08.173019 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5956242c-c9f1-4bc4-82e8-ab9bdf249ca9 service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:08.173478 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5956242c-c9f1-4bc4-82e8-ab9bdf249ca9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:08.173980 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5956242c-c9f1-4bc4-82e8-ab9bdf249ca9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:08.189542 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5956242c-c9f1-4bc4-82e8-ab9bdf249ca9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:08.192060 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5956242c-c9f1-4bc4-82e8-ab9bdf249ca9 service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:08.209366 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1296/2564] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:48:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:48:09.711378 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59c3b18-c009-44a8-bc1a-e6f164cb7112 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:09.712225 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59c3b18-c009-44a8-bc1a-e6f164cb7112 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:09.712225 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59c3b18-c009-44a8-bc1a-e6f164cb7112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:09.730609 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:48:09.741242 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-d59c3b18-c009-44a8-bc1a-e6f164cb7112 None None] MFA Rules not processed for user `10e2d2ab314c4c55a4100384d93ea93f`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:48:09.743875 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d59c3b18-c009-44a8-bc1a-e6f164cb7112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:09.753663 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1269/2565] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:48:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:48:18.201640 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:DELETE=1,keystone:SELECT=55,keystone:INSERT=2 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:48:25.350415 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70894332-65e2-4218-9693-0e62129a84ef None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:25.351100 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70894332-65e2-4218-9693-0e62129a84ef None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:25.351207 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70894332-65e2-4218-9693-0e62129a84ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:25.381605 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-70894332-65e2-4218-9693-0e62129a84ef None None] MFA Rules not processed for user `85d9c4c57681447c898ebfd7990b05b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:48:25.383052 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-70894332-65e2-4218-9693-0e62129a84ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:25.448075 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1297/2566] 10.222.0.13 () {58 vars in 1009 bytes} [Fri Jun 24 08:48:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:48:25.474076 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59c3b18-c009-44a8-bc1a-e6f164cb7112 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:25.475758 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d59c3b18-c009-44a8-bc1a-e6f164cb7112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:25.579557 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59c3b18-c009-44a8-bc1a-e6f164cb7112 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:25.588050 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14d1006-e684-40f0-85f6-ab40250f73ae service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec8c2f113cf41a0939b81f38fbee813', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:25.589156 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d1006-e684-40f0-85f6-ab40250f73ae service swift] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:25.590253 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d1006-e684-40f0-85f6-ab40250f73ae service swift] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:25.590741 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d1006-e684-40f0-85f6-ab40250f73ae service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:25.607961 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14d1006-e684-40f0-85f6-ab40250f73ae service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:25.611137 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14d1006-e684-40f0-85f6-ab40250f73ae service swift] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:25.627336 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1270/2567] 10.222.0.13 () {60 vars in 1384 bytes} [Fri Jun 24 08:48:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 08:48:26.194213 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8131428-1c28-4bf8-aa2f-f396d4ffa6a7 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:26.194511 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8131428-1c28-4bf8-aa2f-f396d4ffa6a7 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:26.194795 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8131428-1c28-4bf8-aa2f-f396d4ffa6a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:26.214437 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-c8131428-1c28-4bf8-aa2f-f396d4ffa6a7 None None] MFA Rules not processed for user `85d9c4c57681447c898ebfd7990b05b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:48:26.216850 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-c8131428-1c28-4bf8-aa2f-f396d4ffa6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:26.225336 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1298/2568] 10.222.0.13 () {58 vars in 967 bytes} [Fri Jun 24 08:48:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:48:26.237614 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14d1006-e684-40f0-85f6-ab40250f73ae service swift] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:26.250384 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14d1006-e684-40f0-85f6-ab40250f73ae service swift] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:26.255684 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7572edf9-0963-4d42-8c04-8f76ce4ef1e3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec8c2f113cf41a0939b81f38fbee813', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:26.256416 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572edf9-0963-4d42-8c04-8f76ce4ef1e3 service swift] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:26.256736 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572edf9-0963-4d42-8c04-8f76ce4ef1e3 service swift] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:26.257048 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572edf9-0963-4d42-8c04-8f76ce4ef1e3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:26.267485 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7572edf9-0963-4d42-8c04-8f76ce4ef1e3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:26.269087 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7572edf9-0963-4d42-8c04-8f76ce4ef1e3 service swift] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:26.287631 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1271/2569] 10.222.0.13 () {60 vars in 1384 bytes} [Fri Jun 24 08:48:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 08:48:26.522262 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a46571-53be-4a33-b552-c9fccf0f18f8 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:26.523109 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a46571-53be-4a33-b552-c9fccf0f18f8 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:26.523109 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a46571-53be-4a33-b552-c9fccf0f18f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:26.546459 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-54a46571-53be-4a33-b552-c9fccf0f18f8 None None] MFA Rules not processed for user `85d9c4c57681447c898ebfd7990b05b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:48:26.547692 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-54a46571-53be-4a33-b552-c9fccf0f18f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:26.555632 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1299/2570] 10.222.0.13 () {58 vars in 1009 bytes} [Fri Jun 24 08:48:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:48:26.568133 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7572edf9-0963-4d42-8c04-8f76ce4ef1e3 service swift] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:26.579967 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7572edf9-0963-4d42-8c04-8f76ce4ef1e3 service swift] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:26.584946 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b29e05-ccef-42ca-903b-aee8aed56d8e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec8c2f113cf41a0939b81f38fbee813', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:26.585584 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b29e05-ccef-42ca-903b-aee8aed56d8e service swift] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:26.585902 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b29e05-ccef-42ca-903b-aee8aed56d8e service swift] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:26.586181 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b29e05-ccef-42ca-903b-aee8aed56d8e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:26.595346 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b29e05-ccef-42ca-903b-aee8aed56d8e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:26.596949 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b29e05-ccef-42ca-903b-aee8aed56d8e service swift] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:26.605927 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1272/2571] 10.222.0.13 () {60 vars in 1384 bytes} [Fri Jun 24 08:48:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 08:48:26.958206 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f54ec-8127-409a-bb10-ef532f34b4c5 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:26.958489 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f54ec-8127-409a-bb10-ef532f34b4c5 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:26.958768 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f54ec-8127-409a-bb10-ef532f34b4c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:26.978564 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-e39f54ec-8127-409a-bb10-ef532f34b4c5 None None] MFA Rules not processed for user `85d9c4c57681447c898ebfd7990b05b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:48:26.979773 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-e39f54ec-8127-409a-bb10-ef532f34b4c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:26.987236 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1300/2572] 10.222.0.13 () {58 vars in 967 bytes} [Fri Jun 24 08:48:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:48:27.001923 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b29e05-ccef-42ca-903b-aee8aed56d8e service swift] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:27.013131 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b29e05-ccef-42ca-903b-aee8aed56d8e service swift] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:27.018210 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d9c448-cbc8-4530-a50b-cb2d38058950 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec8c2f113cf41a0939b81f38fbee813', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:27.019072 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d9c448-cbc8-4530-a50b-cb2d38058950 service swift] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:27.019356 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d9c448-cbc8-4530-a50b-cb2d38058950 service swift] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:27.019632 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d9c448-cbc8-4530-a50b-cb2d38058950 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:27.029295 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d9c448-cbc8-4530-a50b-cb2d38058950 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:27.030943 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d9c448-cbc8-4530-a50b-cb2d38058950 service swift] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:27.039849 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1273/2573] 10.222.0.13 () {60 vars in 1384 bytes} [Fri Jun 24 08:48:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 08:48:29.190316 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39f54ec-8127-409a-bb10-ef532f34b4c5 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:29.270155 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39f54ec-8127-409a-bb10-ef532f34b4c5 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:29.274752 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f030b2ca-7b08-42a3-8459-be3d8f80391f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:29.275634 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030b2ca-7b08-42a3-8459-be3d8f80391f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:29.275910 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030b2ca-7b08-42a3-8459-be3d8f80391f admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:29.276185 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030b2ca-7b08-42a3-8459-be3d8f80391f admin admin] PATH_INFO: `/v3/users/10e2d2ab314c4c55a4100384d93ea93f` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:29.279491 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f030b2ca-7b08-42a3-8459-be3d8f80391f admin admin] RBAC: Authorizing `identity:delete_user(user_id=10e2d2ab314c4c55a4100384d93ea93f)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:29.284569 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f030b2ca-7b08-42a3-8459-be3d8f80391f admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:29.342780 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-f030b2ca-7b08-42a3-8459-be3d8f80391f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10e2d2ab314c4c55a4100384d93ea93f'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:29.358289 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-f030b2ca-7b08-42a3-8459-be3d8f80391f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10e2d2ab314c4c55a4100384d93ea93f'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:29.367153 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-f030b2ca-7b08-42a3-8459-be3d8f80391f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10e2d2ab314c4c55a4100384d93ea93f'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:29.367788 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-f030b2ca-7b08-42a3-8459-be3d8f80391f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10e2d2ab314c4c55a4100384d93ea93f'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:29.368480 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-f030b2ca-7b08-42a3-8459-be3d8f80391f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10e2d2ab314c4c55a4100384d93ea93f'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:29.385647 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1301/2574] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:48:29 2022] DELETE /identity/v3/users/10e2d2ab314c4c55a4100384d93ea93f => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:48:29.391531 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a1484-a71e-40bf-9299-c4d916fec8b2 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:29.392442 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a1484-a71e-40bf-9299-c4d916fec8b2 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:29.392608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a1484-a71e-40bf-9299-c4d916fec8b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:29.413864 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-083a1484-a71e-40bf-9299-c4d916fec8b2 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:48:29.415164 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-083a1484-a71e-40bf-9299-c4d916fec8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:29.502949 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1274/2575] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:48:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:48:29.514076 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f030b2ca-7b08-42a3-8459-be3d8f80391f admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:29.516603 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-f030b2ca-7b08-42a3-8459-be3d8f80391f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:29.523670 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:48:29.587806 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f030b2ca-7b08-42a3-8459-be3d8f80391f admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:29.592961 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc552d8-586c-4716-8338-fbe7d94f2b8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:29.593682 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc552d8-586c-4716-8338-fbe7d94f2b8e service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:29.594046 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc552d8-586c-4716-8338-fbe7d94f2b8e service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:29.594367 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc552d8-586c-4716-8338-fbe7d94f2b8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:29.606010 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc552d8-586c-4716-8338-fbe7d94f2b8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:29.607539 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc552d8-586c-4716-8338-fbe7d94f2b8e service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:29.618822 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1302/2576] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:48:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:48:29.859616 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083a1484-a71e-40bf-9299-c4d916fec8b2 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:29.862087 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-083a1484-a71e-40bf-9299-c4d916fec8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:29.882509 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083a1484-a71e-40bf-9299-c4d916fec8b2 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:29.889285 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc7833-56d3-48ac-9eec-64c22ba20e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:29.890214 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc7833-56d3-48ac-9eec-64c22ba20e17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:29.890520 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc7833-56d3-48ac-9eec-64c22ba20e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:29.890773 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc7833-56d3-48ac-9eec-64c22ba20e17 admin admin] PATH_INFO: `/v3/projects/0d4721b2f8834f2fa81aa7d0d5f090e2` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:29.892099 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efc7833-56d3-48ac-9eec-64c22ba20e17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d4721b2f8834f2fa81aa7d0d5f090e2)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:29.895571 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efc7833-56d3-48ac-9eec-64c22ba20e17 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:29.963898 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:48:29.969751 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-5efc7833-56d3-48ac-9eec-64c22ba20e17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d4721b2f8834f2fa81aa7d0d5f090e2', 'request_id': 'req-5efc7833-56d3-48ac-9eec-64c22ba20e17'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:29.977255 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-5efc7833-56d3-48ac-9eec-64c22ba20e17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d4721b2f8834f2fa81aa7d0d5f090e2', 'request_id': 'req-5efc7833-56d3-48ac-9eec-64c22ba20e17'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:29.984946 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-5efc7833-56d3-48ac-9eec-64c22ba20e17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d4721b2f8834f2fa81aa7d0d5f090e2', 'request_id': 'req-5efc7833-56d3-48ac-9eec-64c22ba20e17'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:30.000081 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-5efc7833-56d3-48ac-9eec-64c22ba20e17 admin admin] The token cache is being invalidate because project 0d4721b2f8834f2fa81aa7d0d5f090e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:48:30.000511 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-5efc7833-56d3-48ac-9eec-64c22ba20e17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:30.001323 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-5efc7833-56d3-48ac-9eec-64c22ba20e17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:30.003487 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1275/2577] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:48:29 2022] DELETE /identity/v3/projects/0d4721b2f8834f2fa81aa7d0d5f090e2 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:48:39.613667 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=24 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:48:39.996385 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:DELETE=4,keystone:SELECT=7,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:48:40.338775 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc552d8-586c-4716-8338-fbe7d94f2b8e service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:40.340974 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-5fc552d8-586c-4716-8338-fbe7d94f2b8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:40.358320 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc552d8-586c-4716-8338-fbe7d94f2b8e service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:40.433943 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4090fbcc-bfda-4ccf-8126-7a94103b439a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:40.434566 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4090fbcc-bfda-4ccf-8126-7a94103b439a service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:40.435005 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4090fbcc-bfda-4ccf-8126-7a94103b439a service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:40.438279 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4090fbcc-bfda-4ccf-8126-7a94103b439a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:40.440809 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-4090fbcc-bfda-4ccf-8126-7a94103b439a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:40.559836 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4090fbcc-bfda-4ccf-8126-7a94103b439a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:40.562490 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4090fbcc-bfda-4ccf-8126-7a94103b439a service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:40.580987 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1303/2578] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:48:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:48:41.881968 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe28f4fa-3362-417c-8eba-ef6e0ac794da None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:41.881968 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe28f4fa-3362-417c-8eba-ef6e0ac794da None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:41.881968 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe28f4fa-3362-417c-8eba-ef6e0ac794da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:41.913566 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-fe28f4fa-3362-417c-8eba-ef6e0ac794da None None] MFA Rules not processed for user `8d15a76041274e7381087bd21eb674c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:48:41.914818 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-fe28f4fa-3362-417c-8eba-ef6e0ac794da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:41.991768 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1276/2579] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:48:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:48:44.735189 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4090fbcc-bfda-4ccf-8126-7a94103b439a service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:44.737339 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-4090fbcc-bfda-4ccf-8126-7a94103b439a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:44.750761 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4090fbcc-bfda-4ccf-8126-7a94103b439a service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:44.755771 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4765b9cb-9035-4dfd-8569-6e68edd2e194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:44.756643 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765b9cb-9035-4dfd-8569-6e68edd2e194 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:44.756928 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765b9cb-9035-4dfd-8569-6e68edd2e194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:44.757205 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765b9cb-9035-4dfd-8569-6e68edd2e194 admin admin] PATH_INFO: `/v3/users/75a612089b2b46a6b3ff5a6cd9e59579` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:44.758417 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4765b9cb-9035-4dfd-8569-6e68edd2e194 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75a612089b2b46a6b3ff5a6cd9e59579)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:44.760335 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4765b9cb-9035-4dfd-8569-6e68edd2e194 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:44.813822 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-4765b9cb-9035-4dfd-8569-6e68edd2e194 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75a612089b2b46a6b3ff5a6cd9e59579'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:44.826533 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-4765b9cb-9035-4dfd-8569-6e68edd2e194 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75a612089b2b46a6b3ff5a6cd9e59579'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:44.835341 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-4765b9cb-9035-4dfd-8569-6e68edd2e194 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75a612089b2b46a6b3ff5a6cd9e59579'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:44.836434 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-4765b9cb-9035-4dfd-8569-6e68edd2e194 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75a612089b2b46a6b3ff5a6cd9e59579'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:44.837377 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-4765b9cb-9035-4dfd-8569-6e68edd2e194 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75a612089b2b46a6b3ff5a6cd9e59579'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:44.861492 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1304/2580] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:48:44 2022] DELETE /identity/v3/users/75a612089b2b46a6b3ff5a6cd9e59579 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:48:45.072975 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe28f4fa-3362-417c-8eba-ef6e0ac794da None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:45.088459 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe28f4fa-3362-417c-8eba-ef6e0ac794da None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:45.182379 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84edfed-1125-424a-9012-e6f9bf840e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:45.184014 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84edfed-1125-424a-9012-e6f9bf840e2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:45.184586 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84edfed-1125-424a-9012-e6f9bf840e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:45.185086 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84edfed-1125-424a-9012-e6f9bf840e2b admin admin] PATH_INFO: `/v3/projects/4de333e57ed34492bee6491335ff4356` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:45.187306 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84edfed-1125-424a-9012-e6f9bf840e2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4de333e57ed34492bee6491335ff4356)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:45.191238 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84edfed-1125-424a-9012-e6f9bf840e2b admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:45.271934 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-a84edfed-1125-424a-9012-e6f9bf840e2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4de333e57ed34492bee6491335ff4356', 'request_id': 'req-a84edfed-1125-424a-9012-e6f9bf840e2b'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:45.280702 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-a84edfed-1125-424a-9012-e6f9bf840e2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4de333e57ed34492bee6491335ff4356', 'request_id': 'req-a84edfed-1125-424a-9012-e6f9bf840e2b'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:45.293031 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-a84edfed-1125-424a-9012-e6f9bf840e2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4de333e57ed34492bee6491335ff4356', 'request_id': 'req-a84edfed-1125-424a-9012-e6f9bf840e2b'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:45.312806 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-a84edfed-1125-424a-9012-e6f9bf840e2b admin admin] The token cache is being invalidate because project 4de333e57ed34492bee6491335ff4356 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:48:45.313357 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-a84edfed-1125-424a-9012-e6f9bf840e2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:45.314031 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-a84edfed-1125-424a-9012-e6f9bf840e2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:45.315581 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1277/2581] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:48:45 2022] DELETE /identity/v3/projects/4de333e57ed34492bee6491335ff4356 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:48:45.320543 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d768d-3313-41b3-9115-57f4a8c757a5 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:45.320859 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d768d-3313-41b3-9115-57f4a8c757a5 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:45.321138 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d768d-3313-41b3-9115-57f4a8c757a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:45.340499 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-df8d768d-3313-41b3-9115-57f4a8c757a5 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:48:45.341722 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-df8d768d-3313-41b3-9115-57f4a8c757a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:45.363971 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:48:45.421979 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1305/2582] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:48:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:48:45.430419 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749184a2-b17b-4797-a1d0-1be221b21ab0 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:45.430704 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749184a2-b17b-4797-a1d0-1be221b21ab0 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:45.430983 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749184a2-b17b-4797-a1d0-1be221b21ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:45.450674 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-749184a2-b17b-4797-a1d0-1be221b21ab0 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:48:45.451921 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-749184a2-b17b-4797-a1d0-1be221b21ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:45.460149 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1278/2583] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:48:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:48:45.464397 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8d768d-3313-41b3-9115-57f4a8c757a5 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:45.475036 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8d768d-3313-41b3-9115-57f4a8c757a5 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:45.481434 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f576443-8f01-4e41-8627-76e16871052c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:45.481434 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f576443-8f01-4e41-8627-76e16871052c admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:45.481434 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f576443-8f01-4e41-8627-76e16871052c admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:45.481804 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f576443-8f01-4e41-8627-76e16871052c admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:45.482379 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f576443-8f01-4e41-8627-76e16871052c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:48:45.482685 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f576443-8f01-4e41-8627-76e16871052c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:45.484288 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f576443-8f01-4e41-8627-76e16871052c admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:45.497812 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1306/2584] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:48:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:48:45.502274 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749184a2-b17b-4797-a1d0-1be221b21ab0 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:45.512478 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749184a2-b17b-4797-a1d0-1be221b21ab0 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:45.517401 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3671169-0f53-48dc-95fa-0549d48e80b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:45.518098 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3671169-0f53-48dc-95fa-0549d48e80b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:45.518388 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3671169-0f53-48dc-95fa-0549d48e80b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:45.518669 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3671169-0f53-48dc-95fa-0549d48e80b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:45.519348 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3671169-0f53-48dc-95fa-0549d48e80b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:45.521265 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3671169-0f53-48dc-95fa-0549d48e80b1 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:45.551293 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1279/2585] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:48:45 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:48:45.555467 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f576443-8f01-4e41-8627-76e16871052c admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:45.567264 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f576443-8f01-4e41-8627-76e16871052c admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:45.644882 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151be57e-d3f3-4091-8967-6aa82b6b7718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:45.645563 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151be57e-d3f3-4091-8967-6aa82b6b7718 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:45.645892 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151be57e-d3f3-4091-8967-6aa82b6b7718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:45.646224 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151be57e-d3f3-4091-8967-6aa82b6b7718 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:45.647006 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151be57e-d3f3-4091-8967-6aa82b6b7718 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:45.649202 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151be57e-d3f3-4091-8967-6aa82b6b7718 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:45.676569 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1307/2586] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:48:45 2022] POST /identity/v3/users => generated 293 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:48:45.681244 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3671169-0f53-48dc-95fa-0549d48e80b1 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:45.707879 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3671169-0f53-48dc-95fa-0549d48e80b1 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:45.716257 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8cb59f-1689-4b1d-b2f9-8ae787376256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:45.717103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8cb59f-1689-4b1d-b2f9-8ae787376256 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:45.717535 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8cb59f-1689-4b1d-b2f9-8ae787376256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:45.718035 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8cb59f-1689-4b1d-b2f9-8ae787376256 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:45.718918 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8cb59f-1689-4b1d-b2f9-8ae787376256 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:45.721269 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8cb59f-1689-4b1d-b2f9-8ae787376256 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:45.729125 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:48:45.738091 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1280/2587] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:48:45 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:48:45.741978 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151be57e-d3f3-4091-8967-6aa82b6b7718 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:45.754978 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151be57e-d3f3-4091-8967-6aa82b6b7718 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:45.760542 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be4ed3e-c9d3-4a41-ac57-50977cf76251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:45.761537 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be4ed3e-c9d3-4a41-ac57-50977cf76251 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:45.761872 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be4ed3e-c9d3-4a41-ac57-50977cf76251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:45.762096 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be4ed3e-c9d3-4a41-ac57-50977cf76251 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:45.763056 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be4ed3e-c9d3-4a41-ac57-50977cf76251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:45.764684 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be4ed3e-c9d3-4a41-ac57-50977cf76251 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:45.779223 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1308/2588] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:48:45 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:48:45.783614 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8cb59f-1689-4b1d-b2f9-8ae787376256 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:45.795635 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8cb59f-1689-4b1d-b2f9-8ae787376256 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:45.801035 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7018490c-9d60-43f8-bd37-0c7aead09a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:45.801982 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7018490c-9d60-43f8-bd37-0c7aead09a54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:45.802284 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7018490c-9d60-43f8-bd37-0c7aead09a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:45.802558 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7018490c-9d60-43f8-bd37-0c7aead09a54 admin admin] PATH_INFO: `/v3/projects/96de0530d58a41869b79bec262b795bd/users/c37d325f859f47c48fbe28835b0ed631/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:45.823560 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7018490c-9d60-43f8-bd37-0c7aead09a54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96de0530d58a41869b79bec262b795bd, user_id=c37d325f859f47c48fbe28835b0ed631, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:45.829649 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7018490c-9d60-43f8-bd37-0c7aead09a54 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:45.857756 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1281/2589] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:48:45 2022] PUT /identity/v3/projects/96de0530d58a41869b79bec262b795bd/users/c37d325f859f47c48fbe28835b0ed631/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:48:45.869834 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6308d613-5967-452e-8df3-1cf033d3eec1 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:45.870322 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6308d613-5967-452e-8df3-1cf033d3eec1 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:45.871671 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6308d613-5967-452e-8df3-1cf033d3eec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:45.905564 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-6308d613-5967-452e-8df3-1cf033d3eec1 None None] MFA Rules not processed for user `c37d325f859f47c48fbe28835b0ed631`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:48:45.906758 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-6308d613-5967-452e-8df3-1cf033d3eec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:45.996294 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1309/2590] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:48:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:48:45.997054 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7018490c-9d60-43f8-bd37-0c7aead09a54 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:46.081314 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7018490c-9d60-43f8-bd37-0c7aead09a54 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:46.086207 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f49a3-cc7c-4d68-9444-04fe055e7ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:46.087193 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f49a3-cc7c-4d68-9444-04fe055e7ead admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:46.087486 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f49a3-cc7c-4d68-9444-04fe055e7ead admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:46.087765 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f49a3-cc7c-4d68-9444-04fe055e7ead admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:46.088459 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9f49a3-cc7c-4d68-9444-04fe055e7ead admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:46.091303 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9f49a3-cc7c-4d68-9444-04fe055e7ead admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:46.121538 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1282/2591] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:48:45 2022] POST /identity/v3/projects => generated 385 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:48:46.126276 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6308d613-5967-452e-8df3-1cf033d3eec1 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:46.139724 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6308d613-5967-452e-8df3-1cf033d3eec1 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:46.154721 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:48:46.208369 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0ef775-f247-4d9d-9580-85de5a57772a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:46.209388 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ef775-f247-4d9d-9580-85de5a57772a admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:46.209812 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ef775-f247-4d9d-9580-85de5a57772a admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:46.210169 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ef775-f247-4d9d-9580-85de5a57772a admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:46.210934 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0ef775-f247-4d9d-9580-85de5a57772a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:46.213467 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0ef775-f247-4d9d-9580-85de5a57772a admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:46.246604 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1310/2592] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:48:46 2022] POST /identity/v3/users => generated 299 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:48:46.248027 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f49a3-cc7c-4d68-9444-04fe055e7ead admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:46.266577 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f49a3-cc7c-4d68-9444-04fe055e7ead admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:46.275640 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e52212a-a603-4b71-b223-9beace93e58d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:46.277347 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52212a-a603-4b71-b223-9beace93e58d admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:46.277774 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52212a-a603-4b71-b223-9beace93e58d admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:46.278294 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52212a-a603-4b71-b223-9beace93e58d admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:46.279260 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e52212a-a603-4b71-b223-9beace93e58d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:46.280694 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e52212a-a603-4b71-b223-9beace93e58d admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:46.294356 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1283/2593] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:48:46 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:48:46.299029 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0ef775-f247-4d9d-9580-85de5a57772a admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:46.311115 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0ef775-f247-4d9d-9580-85de5a57772a admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:46.316959 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f032cd-b65f-44a6-a680-755748e9d917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:46.317925 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f032cd-b65f-44a6-a680-755748e9d917 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:46.318213 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f032cd-b65f-44a6-a680-755748e9d917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:46.318494 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f032cd-b65f-44a6-a680-755748e9d917 admin admin] PATH_INFO: `/v3/projects/06973281c84444c5b880258a81fff1cf/users/f24f79635b964005880454f7c17ff9bb/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:46.337410 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f032cd-b65f-44a6-a680-755748e9d917 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06973281c84444c5b880258a81fff1cf, user_id=f24f79635b964005880454f7c17ff9bb, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:46.341003 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f032cd-b65f-44a6-a680-755748e9d917 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:46.359289 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1311/2594] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:48:46 2022] PUT /identity/v3/projects/06973281c84444c5b880258a81fff1cf/users/f24f79635b964005880454f7c17ff9bb/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:48:46.360801 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e52212a-a603-4b71-b223-9beace93e58d admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:46.373122 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e52212a-a603-4b71-b223-9beace93e58d admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:46.450203 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b95327-e8b6-4bb7-9ca8-8f4084d71333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:46.450980 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b95327-e8b6-4bb7-9ca8-8f4084d71333 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:46.451391 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b95327-e8b6-4bb7-9ca8-8f4084d71333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:46.451733 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b95327-e8b6-4bb7-9ca8-8f4084d71333 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:46.454215 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b95327-e8b6-4bb7-9ca8-8f4084d71333 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:46.454430 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b95327-e8b6-4bb7-9ca8-8f4084d71333 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:46.468198 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1284/2595] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:48:46 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:48:46.472307 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f032cd-b65f-44a6-a680-755748e9d917 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:46.484379 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f032cd-b65f-44a6-a680-755748e9d917 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:46.489758 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273515b3-1233-46a4-b37d-b6c267ab1aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:46.490676 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273515b3-1233-46a4-b37d-b6c267ab1aef admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:46.491029 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273515b3-1233-46a4-b37d-b6c267ab1aef admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:46.491373 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273515b3-1233-46a4-b37d-b6c267ab1aef admin admin] PATH_INFO: `/v3/projects/06973281c84444c5b880258a81fff1cf/users/f24f79635b964005880454f7c17ff9bb/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:46.493620 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273515b3-1233-46a4-b37d-b6c267ab1aef admin admin] RBAC: Authorizing `identity:create_grant(project_id=06973281c84444c5b880258a81fff1cf, user_id=f24f79635b964005880454f7c17ff9bb, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:46.497403 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273515b3-1233-46a4-b37d-b6c267ab1aef admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:46.512163 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1312/2596] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:48:46 2022] PUT /identity/v3/projects/06973281c84444c5b880258a81fff1cf/users/f24f79635b964005880454f7c17ff9bb/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:48:46.533120 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa2a903-0d3e-4351-9eeb-18ac8071f3c2 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:46.533773 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa2a903-0d3e-4351-9eeb-18ac8071f3c2 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:46.534345 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa2a903-0d3e-4351-9eeb-18ac8071f3c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:46.582101 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-baa2a903-0d3e-4351-9eeb-18ac8071f3c2 None None] MFA Rules not processed for user `f24f79635b964005880454f7c17ff9bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:48:46.586624 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-baa2a903-0d3e-4351-9eeb-18ac8071f3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:46.632356 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:48:46.725344 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1285/2597] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:48:46 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:48:46.736380 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273515b3-1233-46a4-b37d-b6c267ab1aef admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:46.739086 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-273515b3-1233-46a4-b37d-b6c267ab1aef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:46.837599 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273515b3-1233-46a4-b37d-b6c267ab1aef admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:46.844796 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6076613a-b905-476a-88e5-d2d94e326c8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:46.846013 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076613a-b905-476a-88e5-d2d94e326c8b service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:46.846521 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076613a-b905-476a-88e5-d2d94e326c8b service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:46.847025 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076613a-b905-476a-88e5-d2d94e326c8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:46.865657 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6076613a-b905-476a-88e5-d2d94e326c8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:46.867782 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6076613a-b905-476a-88e5-d2d94e326c8b service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:46.892798 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1313/2598] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:48:46 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:48:46.945194 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa2a903-0d3e-4351-9eeb-18ac8071f3c2 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:47.041222 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa2a903-0d3e-4351-9eeb-18ac8071f3c2 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:47.047716 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c349727-a20e-40a8-92b2-a6d95e825656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:47.048643 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c349727-a20e-40a8-92b2-a6d95e825656 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:47.048911 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c349727-a20e-40a8-92b2-a6d95e825656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:47.049194 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c349727-a20e-40a8-92b2-a6d95e825656 admin admin] PATH_INFO: `/v3/users/c37d325f859f47c48fbe28835b0ed631` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:47.050708 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c349727-a20e-40a8-92b2-a6d95e825656 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c37d325f859f47c48fbe28835b0ed631)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:47.052923 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c349727-a20e-40a8-92b2-a6d95e825656 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:47.117840 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-4c349727-a20e-40a8-92b2-a6d95e825656 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c37d325f859f47c48fbe28835b0ed631'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:47.145725 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-4c349727-a20e-40a8-92b2-a6d95e825656 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c37d325f859f47c48fbe28835b0ed631'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:47.147248 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-4c349727-a20e-40a8-92b2-a6d95e825656 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c37d325f859f47c48fbe28835b0ed631'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:47.169721 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-4c349727-a20e-40a8-92b2-a6d95e825656 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c37d325f859f47c48fbe28835b0ed631'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:47.170572 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-4c349727-a20e-40a8-92b2-a6d95e825656 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c37d325f859f47c48fbe28835b0ed631'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:47.190232 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1286/2599] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:48:46 2022] DELETE /identity/v3/users/c37d325f859f47c48fbe28835b0ed631 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:48:47.196416 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddfa65b-a26b-410d-8478-9a1637979f64 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:47.197080 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddfa65b-a26b-410d-8478-9a1637979f64 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:47.197646 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddfa65b-a26b-410d-8478-9a1637979f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:47.221464 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-eddfa65b-a26b-410d-8478-9a1637979f64 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:48:47.222754 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-eddfa65b-a26b-410d-8478-9a1637979f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:47.300650 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1314/2600] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:48:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:48:47.313659 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c349727-a20e-40a8-92b2-a6d95e825656 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:47.315326 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-4c349727-a20e-40a8-92b2-a6d95e825656 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:47.391852 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:48:47.400155 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c349727-a20e-40a8-92b2-a6d95e825656 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:47.407330 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f6456-e54e-4365-a8ab-33aa93184595 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:47.408238 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f6456-e54e-4365-a8ab-33aa93184595 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:47.408704 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f6456-e54e-4365-a8ab-33aa93184595 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:47.409250 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f6456-e54e-4365-a8ab-33aa93184595 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:47.424293 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09f6456-e54e-4365-a8ab-33aa93184595 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:47.426534 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09f6456-e54e-4365-a8ab-33aa93184595 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:47.439330 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1287/2601] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:48:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:48:47.757058 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddfa65b-a26b-410d-8478-9a1637979f64 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:47.759055 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-eddfa65b-a26b-410d-8478-9a1637979f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:47.780100 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddfa65b-a26b-410d-8478-9a1637979f64 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:47.787964 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c35fbd5-6811-4c4a-b257-6a5dd7bdc89b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:47.789120 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c35fbd5-6811-4c4a-b257-6a5dd7bdc89b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:47.789579 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c35fbd5-6811-4c4a-b257-6a5dd7bdc89b admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:47.789886 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c35fbd5-6811-4c4a-b257-6a5dd7bdc89b admin admin] PATH_INFO: `/v3/projects/96de0530d58a41869b79bec262b795bd` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:47.791241 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c35fbd5-6811-4c4a-b257-6a5dd7bdc89b admin admin] RBAC: Authorizing `identity:delete_project(project_id=96de0530d58a41869b79bec262b795bd)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:47.794028 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c35fbd5-6811-4c4a-b257-6a5dd7bdc89b admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:47.804540 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:48:47.871807 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-2c35fbd5-6811-4c4a-b257-6a5dd7bdc89b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96de0530d58a41869b79bec262b795bd', 'request_id': 'req-2c35fbd5-6811-4c4a-b257-6a5dd7bdc89b'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:47.883228 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-2c35fbd5-6811-4c4a-b257-6a5dd7bdc89b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96de0530d58a41869b79bec262b795bd', 'request_id': 'req-2c35fbd5-6811-4c4a-b257-6a5dd7bdc89b'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:47.894343 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-2c35fbd5-6811-4c4a-b257-6a5dd7bdc89b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96de0530d58a41869b79bec262b795bd', 'request_id': 'req-2c35fbd5-6811-4c4a-b257-6a5dd7bdc89b'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:47.919360 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-2c35fbd5-6811-4c4a-b257-6a5dd7bdc89b admin admin] The token cache is being invalidate because project 96de0530d58a41869b79bec262b795bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:48:47.919825 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-2c35fbd5-6811-4c4a-b257-6a5dd7bdc89b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:47.920779 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-2c35fbd5-6811-4c4a-b257-6a5dd7bdc89b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:47.922732 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1315/2602] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:48:47 2022] DELETE /identity/v3/projects/96de0530d58a41869b79bec262b795bd => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:48:47.927229 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f6456-e54e-4365-a8ab-33aa93184595 service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:47.940966 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f6456-e54e-4365-a8ab-33aa93184595 service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:48.012565 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3535affd-7d47-41a4-9f74-b67b323be98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:48.013292 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3535affd-7d47-41a4-9f74-b67b323be98a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:48.013582 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3535affd-7d47-41a4-9f74-b67b323be98a admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:48.013898 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3535affd-7d47-41a4-9f74-b67b323be98a admin admin] PATH_INFO: `/v3/users/f24f79635b964005880454f7c17ff9bb` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:48.015178 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3535affd-7d47-41a4-9f74-b67b323be98a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f24f79635b964005880454f7c17ff9bb)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:48.017119 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3535affd-7d47-41a4-9f74-b67b323be98a admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:48.071121 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-3535affd-7d47-41a4-9f74-b67b323be98a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f24f79635b964005880454f7c17ff9bb'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:48.083626 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-3535affd-7d47-41a4-9f74-b67b323be98a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f24f79635b964005880454f7c17ff9bb'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:48.084520 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-3535affd-7d47-41a4-9f74-b67b323be98a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f24f79635b964005880454f7c17ff9bb'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:48.092883 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-3535affd-7d47-41a4-9f74-b67b323be98a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f24f79635b964005880454f7c17ff9bb'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:48.093610 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-3535affd-7d47-41a4-9f74-b67b323be98a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f24f79635b964005880454f7c17ff9bb'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:48.108654 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1288/2603] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:48:47 2022] DELETE /identity/v3/users/f24f79635b964005880454f7c17ff9bb => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:48:48.412802 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c35fbd5-6811-4c4a-b257-6a5dd7bdc89b admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:48.414294 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-2c35fbd5-6811-4c4a-b257-6a5dd7bdc89b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:48.438550 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c35fbd5-6811-4c4a-b257-6a5dd7bdc89b admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:48.513581 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50119f28-62b9-4ea7-9d1c-eaf4eba50ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:48.514495 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50119f28-62b9-4ea7-9d1c-eaf4eba50ea2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:48.514780 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50119f28-62b9-4ea7-9d1c-eaf4eba50ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:48.515059 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50119f28-62b9-4ea7-9d1c-eaf4eba50ea2 admin admin] PATH_INFO: `/v3/projects/06973281c84444c5b880258a81fff1cf` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:48.516310 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50119f28-62b9-4ea7-9d1c-eaf4eba50ea2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06973281c84444c5b880258a81fff1cf)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:48.518266 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50119f28-62b9-4ea7-9d1c-eaf4eba50ea2 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:48.583723 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-50119f28-62b9-4ea7-9d1c-eaf4eba50ea2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06973281c84444c5b880258a81fff1cf', 'request_id': 'req-50119f28-62b9-4ea7-9d1c-eaf4eba50ea2'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:48.591985 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-50119f28-62b9-4ea7-9d1c-eaf4eba50ea2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06973281c84444c5b880258a81fff1cf', 'request_id': 'req-50119f28-62b9-4ea7-9d1c-eaf4eba50ea2'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:48.598499 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-50119f28-62b9-4ea7-9d1c-eaf4eba50ea2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06973281c84444c5b880258a81fff1cf', 'request_id': 'req-50119f28-62b9-4ea7-9d1c-eaf4eba50ea2'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:48.613952 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-50119f28-62b9-4ea7-9d1c-eaf4eba50ea2 admin admin] The token cache is being invalidate because project 06973281c84444c5b880258a81fff1cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:48:48.614606 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-50119f28-62b9-4ea7-9d1c-eaf4eba50ea2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:48.615654 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-50119f28-62b9-4ea7-9d1c-eaf4eba50ea2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:48.618277 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1316/2604] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:48:48 2022] DELETE /identity/v3/projects/06973281c84444c5b880258a81fff1cf => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:48:48.625130 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6215ab-4d09-44cd-b549-ca02a271544a None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:48.625658 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6215ab-4d09-44cd-b549-ca02a271544a None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:48.626046 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6215ab-4d09-44cd-b549-ca02a271544a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:48.646649 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-6d6215ab-4d09-44cd-b549-ca02a271544a None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:48:48.647896 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-6d6215ab-4d09-44cd-b549-ca02a271544a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:48.721529 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1289/2605] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:48:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:48:48.731947 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe800df4-f321-4fe5-a55c-2a34c57c80ff None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:48.732337 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe800df4-f321-4fe5-a55c-2a34c57c80ff None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:48.732690 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe800df4-f321-4fe5-a55c-2a34c57c80ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:48.752702 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-fe800df4-f321-4fe5-a55c-2a34c57c80ff None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:48:48.754234 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-fe800df4-f321-4fe5-a55c-2a34c57c80ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:48.762799 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1317/2606] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:48:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:48:48.767511 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6215ab-4d09-44cd-b549-ca02a271544a None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:48.778641 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6215ab-4d09-44cd-b549-ca02a271544a None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:48.784306 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801e8c5e-6e37-4061-a0aa-b9199d1c113b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:48.785233 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e8c5e-6e37-4061-a0aa-b9199d1c113b admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:48.785820 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e8c5e-6e37-4061-a0aa-b9199d1c113b admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:48.786251 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e8c5e-6e37-4061-a0aa-b9199d1c113b admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:48.787207 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801e8c5e-6e37-4061-a0aa-b9199d1c113b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:48:48.787776 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801e8c5e-6e37-4061-a0aa-b9199d1c113b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:48.789981 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801e8c5e-6e37-4061-a0aa-b9199d1c113b admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:48.794131 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:48:48.810146 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1290/2607] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:48:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:48:48.810417 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe800df4-f321-4fe5-a55c-2a34c57c80ff None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:48.820021 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe800df4-f321-4fe5-a55c-2a34c57c80ff None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:48.824791 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63d232b-b8ee-44c1-9485-2297254f837f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:48.825454 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d232b-b8ee-44c1-9485-2297254f837f admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:48.826186 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d232b-b8ee-44c1-9485-2297254f837f admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:48.826186 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d232b-b8ee-44c1-9485-2297254f837f admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:48.826794 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63d232b-b8ee-44c1-9485-2297254f837f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:48.828707 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63d232b-b8ee-44c1-9485-2297254f837f admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:48.854674 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:48:48.865025 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1318/2608] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:48:48 2022] POST /identity/v3/projects => generated 387 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:48:48.869430 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801e8c5e-6e37-4061-a0aa-b9199d1c113b admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:48.881359 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801e8c5e-6e37-4061-a0aa-b9199d1c113b admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:48.951279 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2311adb-c46f-42bc-8968-529644e44a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:48.952115 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2311adb-c46f-42bc-8968-529644e44a6e admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:48.952491 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2311adb-c46f-42bc-8968-529644e44a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:48.952876 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2311adb-c46f-42bc-8968-529644e44a6e admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:48.953732 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2311adb-c46f-42bc-8968-529644e44a6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:48.955961 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2311adb-c46f-42bc-8968-529644e44a6e admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:48.964669 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63d232b-b8ee-44c1-9485-2297254f837f admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:48.966777 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-f63d232b-b8ee-44c1-9485-2297254f837f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:48.988467 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1291/2609] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:48:48 2022] POST /identity/v3/users => generated 294 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:48:48.989608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2311adb-c46f-42bc-8968-529644e44a6e admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:49.001199 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2311adb-c46f-42bc-8968-529644e44a6e admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:49.008906 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cd9573-4668-44ce-b42e-2d37a3c87816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:49.009589 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd9573-4668-44ce-b42e-2d37a3c87816 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:49.009920 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd9573-4668-44ce-b42e-2d37a3c87816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:49.010198 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd9573-4668-44ce-b42e-2d37a3c87816 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:49.010959 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cd9573-4668-44ce-b42e-2d37a3c87816 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:49.013556 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cd9573-4668-44ce-b42e-2d37a3c87816 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:49.032970 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1292/2610] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:48:48 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:48:49.037424 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cd9573-4668-44ce-b42e-2d37a3c87816 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:49.063205 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cd9573-4668-44ce-b42e-2d37a3c87816 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:49.072723 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63d232b-b8ee-44c1-9485-2297254f837f admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:49.074918 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55aa70b-3aba-4b0d-9bb8-df39bbfb3457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:49.075665 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55aa70b-3aba-4b0d-9bb8-df39bbfb3457 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:49.076125 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55aa70b-3aba-4b0d-9bb8-df39bbfb3457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:49.076512 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55aa70b-3aba-4b0d-9bb8-df39bbfb3457 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:49.077339 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55aa70b-3aba-4b0d-9bb8-df39bbfb3457 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:49.079084 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55aa70b-3aba-4b0d-9bb8-df39bbfb3457 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:49.083800 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9812d58-76cb-4498-afe8-2e33e14f03bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:49.087680 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9812d58-76cb-4498-afe8-2e33e14f03bd service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:49.088283 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9812d58-76cb-4498-afe8-2e33e14f03bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:49.088925 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9812d58-76cb-4498-afe8-2e33e14f03bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:49.092348 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-a9812d58-76cb-4498-afe8-2e33e14f03bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:49.096264 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1293/2611] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:48:49 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:48:49.102525 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55aa70b-3aba-4b0d-9bb8-df39bbfb3457 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:49.114984 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55aa70b-3aba-4b0d-9bb8-df39bbfb3457 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:49.122265 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804afd49-e2ba-41fc-9466-34b638dc048b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:49.123429 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804afd49-e2ba-41fc-9466-34b638dc048b admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:49.123836 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804afd49-e2ba-41fc-9466-34b638dc048b admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:49.124316 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804afd49-e2ba-41fc-9466-34b638dc048b admin admin] PATH_INFO: `/v3/projects/e0e5a32a8f674a0aa5aa91bb83b14233/users/9bb840d3901f459daf30082af46d3841/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:49.150158 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804afd49-e2ba-41fc-9466-34b638dc048b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0e5a32a8f674a0aa5aa91bb83b14233, user_id=9bb840d3901f459daf30082af46d3841, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:49.153631 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804afd49-e2ba-41fc-9466-34b638dc048b admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:49.174937 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1294/2612] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:48:49 2022] PUT /identity/v3/projects/e0e5a32a8f674a0aa5aa91bb83b14233/users/9bb840d3901f459daf30082af46d3841/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:48:49.186679 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a61dbf-6982-4138-9575-6aee88f296c5 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:49.187067 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a61dbf-6982-4138-9575-6aee88f296c5 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:49.187421 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a61dbf-6982-4138-9575-6aee88f296c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:49.223076 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-01a61dbf-6982-4138-9575-6aee88f296c5 None None] MFA Rules not processed for user `9bb840d3901f459daf30082af46d3841`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:48:49.224504 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-01a61dbf-6982-4138-9575-6aee88f296c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:49.225731 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9812d58-76cb-4498-afe8-2e33e14f03bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:49.228828 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9812d58-76cb-4498-afe8-2e33e14f03bd service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:49.248897 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1319/2613] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:48:48 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:48:49.360587 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1295/2614] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:48:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:48:49.366420 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9812d58-76cb-4498-afe8-2e33e14f03bd service cinder] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:49.384433 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9812d58-76cb-4498-afe8-2e33e14f03bd service cinder] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:49.484910 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5247cfa-c184-4d60-aa6b-642371825074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:49.486425 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5247cfa-c184-4d60-aa6b-642371825074 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:49.486930 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5247cfa-c184-4d60-aa6b-642371825074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:49.487422 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5247cfa-c184-4d60-aa6b-642371825074 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:49.488774 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5247cfa-c184-4d60-aa6b-642371825074 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:49.492785 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5247cfa-c184-4d60-aa6b-642371825074 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:49.534272 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1320/2615] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:48:49 2022] POST /identity/v3/projects => generated 387 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:48:49.539103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a61dbf-6982-4138-9575-6aee88f296c5 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:49.553217 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a61dbf-6982-4138-9575-6aee88f296c5 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:49.585383 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:48:49.632594 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d684fb93-e3c6-492f-9d19-b274b03f2352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:49.633528 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d684fb93-e3c6-492f-9d19-b274b03f2352 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:49.633740 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d684fb93-e3c6-492f-9d19-b274b03f2352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:49.633940 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d684fb93-e3c6-492f-9d19-b274b03f2352 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:49.634670 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d684fb93-e3c6-492f-9d19-b274b03f2352 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:49.637981 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d684fb93-e3c6-492f-9d19-b274b03f2352 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:49.664309 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1296/2616] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:48:49 2022] POST /identity/v3/users => generated 294 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:48:49.666218 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5247cfa-c184-4d60-aa6b-642371825074 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:49.679098 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5247cfa-c184-4d60-aa6b-642371825074 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:49.684756 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d9b53f-8866-4a90-a28e-1cefd8eac8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:49.685447 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d9b53f-8866-4a90-a28e-1cefd8eac8af admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:49.685762 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d9b53f-8866-4a90-a28e-1cefd8eac8af admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:49.686045 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d9b53f-8866-4a90-a28e-1cefd8eac8af admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:49.686789 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d9b53f-8866-4a90-a28e-1cefd8eac8af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:49.688416 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d9b53f-8866-4a90-a28e-1cefd8eac8af admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:49.701447 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1321/2617] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:48:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:48:49.705616 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d684fb93-e3c6-492f-9d19-b274b03f2352 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:49.717061 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d684fb93-e3c6-492f-9d19-b274b03f2352 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:49.725209 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4904c29f-173f-44d1-a537-a0a9ce384e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:49.726584 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4904c29f-173f-44d1-a537-a0a9ce384e01 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:49.727152 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4904c29f-173f-44d1-a537-a0a9ce384e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:49.727657 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4904c29f-173f-44d1-a537-a0a9ce384e01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:49.729080 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4904c29f-173f-44d1-a537-a0a9ce384e01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:49.732878 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4904c29f-173f-44d1-a537-a0a9ce384e01 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:49.762139 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1297/2618] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:48:49 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:48:49.766060 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d9b53f-8866-4a90-a28e-1cefd8eac8af admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:49.778976 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d9b53f-8866-4a90-a28e-1cefd8eac8af admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:49.785194 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90cd842-64be-4b50-b491-41f03fda211e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:49.786283 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90cd842-64be-4b50-b491-41f03fda211e admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:49.786563 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90cd842-64be-4b50-b491-41f03fda211e admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:49.786835 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90cd842-64be-4b50-b491-41f03fda211e admin admin] PATH_INFO: `/v3/projects/5b2f50f418e14da68a6ed374ca5d8e78/users/d44daccb967e47b1a5b15b5f574b38ac/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:49.803950 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90cd842-64be-4b50-b491-41f03fda211e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b2f50f418e14da68a6ed374ca5d8e78, user_id=d44daccb967e47b1a5b15b5f574b38ac, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:49.806691 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90cd842-64be-4b50-b491-41f03fda211e admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:49.822606 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1322/2619] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:48:49 2022] PUT /identity/v3/projects/5b2f50f418e14da68a6ed374ca5d8e78/users/d44daccb967e47b1a5b15b5f574b38ac/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:48:49.833854 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a5eb86-83a3-4840-86f9-6f7ef2411bed None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:49.834201 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a5eb86-83a3-4840-86f9-6f7ef2411bed None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:49.834526 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a5eb86-83a3-4840-86f9-6f7ef2411bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:49.869667 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-f0a5eb86-83a3-4840-86f9-6f7ef2411bed None None] MFA Rules not processed for user `d44daccb967e47b1a5b15b5f574b38ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:48:49.871569 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-f0a5eb86-83a3-4840-86f9-6f7ef2411bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:49.951269 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1298/2620] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:48:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:48:49.956016 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90cd842-64be-4b50-b491-41f03fda211e admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:49.960810 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:48:50.036829 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90cd842-64be-4b50-b491-41f03fda211e admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:50.041869 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0d9b3b-cf09-4262-a967-83475acdfde1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:50.042586 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d9b3b-cf09-4262-a967-83475acdfde1 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:50.042882 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d9b3b-cf09-4262-a967-83475acdfde1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:50.043161 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d9b3b-cf09-4262-a967-83475acdfde1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:50.043870 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0d9b3b-cf09-4262-a967-83475acdfde1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:50.045827 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0d9b3b-cf09-4262-a967-83475acdfde1 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:50.076364 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1323/2621] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:48:49 2022] POST /identity/v3/projects => generated 381 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:48:50.082871 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a5eb86-83a3-4840-86f9-6f7ef2411bed None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:50.103630 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a5eb86-83a3-4840-86f9-6f7ef2411bed None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:50.170968 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a53642c-5921-447d-893f-46b2b4f13d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:50.171667 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a53642c-5921-447d-893f-46b2b4f13d54 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:50.171950 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a53642c-5921-447d-893f-46b2b4f13d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:50.172223 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a53642c-5921-447d-893f-46b2b4f13d54 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:50.172951 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a53642c-5921-447d-893f-46b2b4f13d54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:50.175013 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a53642c-5921-447d-893f-46b2b4f13d54 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:50.198300 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1299/2622] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:48:50 2022] POST /identity/v3/users => generated 297 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:48:50.203699 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0d9b3b-cf09-4262-a967-83475acdfde1 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:50.215403 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0d9b3b-cf09-4262-a967-83475acdfde1 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:50.221086 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9b61af-365c-4136-bc02-df34eeed919e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:50.221865 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b61af-365c-4136-bc02-df34eeed919e admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:50.222153 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b61af-365c-4136-bc02-df34eeed919e admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:50.222432 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b61af-365c-4136-bc02-df34eeed919e admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:50.223188 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9b61af-365c-4136-bc02-df34eeed919e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:50.224872 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9b61af-365c-4136-bc02-df34eeed919e admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:50.237409 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1324/2623] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:48:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:48:50.242079 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a53642c-5921-447d-893f-46b2b4f13d54 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:50.253926 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a53642c-5921-447d-893f-46b2b4f13d54 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:50.258901 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170e2c4d-1543-4de9-a12d-911bb4b41679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:50.259842 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170e2c4d-1543-4de9-a12d-911bb4b41679 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:50.260122 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170e2c4d-1543-4de9-a12d-911bb4b41679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:50.260398 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170e2c4d-1543-4de9-a12d-911bb4b41679 admin admin] PATH_INFO: `/v3/projects/640091cda2bd4938a1cfa42c70f1c3b8/users/06177ec4673f461d8a8d7baf2f611ba7/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:50.279813 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170e2c4d-1543-4de9-a12d-911bb4b41679 admin admin] RBAC: Authorizing `identity:create_grant(project_id=640091cda2bd4938a1cfa42c70f1c3b8, user_id=06177ec4673f461d8a8d7baf2f611ba7, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:50.282703 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170e2c4d-1543-4de9-a12d-911bb4b41679 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:50.300943 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1300/2624] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:48:50 2022] PUT /identity/v3/projects/640091cda2bd4938a1cfa42c70f1c3b8/users/06177ec4673f461d8a8d7baf2f611ba7/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:48:50.305504 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9b61af-365c-4136-bc02-df34eeed919e admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:50.326028 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9b61af-365c-4136-bc02-df34eeed919e admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:50.421329 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b11674-15be-4b70-85c0-05805b823ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:50.422054 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b11674-15be-4b70-85c0-05805b823ff3 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:50.422340 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b11674-15be-4b70-85c0-05805b823ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:50.422610 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b11674-15be-4b70-85c0-05805b823ff3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:50.423349 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b11674-15be-4b70-85c0-05805b823ff3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:50.424969 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b11674-15be-4b70-85c0-05805b823ff3 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:50.436424 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1325/2625] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:48:50 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:48:50.440864 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170e2c4d-1543-4de9-a12d-911bb4b41679 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:50.446708 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:48:50.455664 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170e2c4d-1543-4de9-a12d-911bb4b41679 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:50.464461 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6c84d3-ed22-4ee3-a6b9-d7004caab328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:50.465371 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c84d3-ed22-4ee3-a6b9-d7004caab328 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:50.465883 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c84d3-ed22-4ee3-a6b9-d7004caab328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:50.466164 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c84d3-ed22-4ee3-a6b9-d7004caab328 admin admin] PATH_INFO: `/v3/projects/640091cda2bd4938a1cfa42c70f1c3b8/users/06177ec4673f461d8a8d7baf2f611ba7/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:50.468384 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6c84d3-ed22-4ee3-a6b9-d7004caab328 admin admin] RBAC: Authorizing `identity:create_grant(project_id=640091cda2bd4938a1cfa42c70f1c3b8, user_id=06177ec4673f461d8a8d7baf2f611ba7, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:50.471128 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6c84d3-ed22-4ee3-a6b9-d7004caab328 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:50.485057 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1301/2626] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:48:50 2022] PUT /identity/v3/projects/640091cda2bd4938a1cfa42c70f1c3b8/users/06177ec4673f461d8a8d7baf2f611ba7/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:48:50.505453 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d07210-89dd-471b-9d46-18f644a28a31 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:50.506075 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d07210-89dd-471b-9d46-18f644a28a31 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:50.506585 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d07210-89dd-471b-9d46-18f644a28a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:50.550891 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-71d07210-89dd-471b-9d46-18f644a28a31 None None] MFA Rules not processed for user `06177ec4673f461d8a8d7baf2f611ba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:48:50.552534 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-71d07210-89dd-471b-9d46-18f644a28a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:50.628456 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:48:50.635399 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1326/2627] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:48:50 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:48:50.646806 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6c84d3-ed22-4ee3-a6b9-d7004caab328 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:50.648729 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-6e6c84d3-ed22-4ee3-a6b9-d7004caab328 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:50.711182 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6c84d3-ed22-4ee3-a6b9-d7004caab328 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:50.715476 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf3e23d-43bf-4535-a124-eb8678935c6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:50.716134 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf3e23d-43bf-4535-a124-eb8678935c6a service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:50.716423 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf3e23d-43bf-4535-a124-eb8678935c6a service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:50.716702 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf3e23d-43bf-4535-a124-eb8678935c6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:50.726997 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf3e23d-43bf-4535-a124-eb8678935c6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:50.728450 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf3e23d-43bf-4535-a124-eb8678935c6a service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:50.739506 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1302/2628] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:48:50 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:48:51.368574 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d07210-89dd-471b-9d46-18f644a28a31 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:51.379303 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d07210-89dd-471b-9d46-18f644a28a31 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:51.383220 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbbe711-d2c0-4d0c-a9d6-4a58e00ee747 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:51.383835 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbbe711-d2c0-4d0c-a9d6-4a58e00ee747 service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:51.384114 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbbe711-d2c0-4d0c-a9d6-4a58e00ee747 service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:51.384386 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbbe711-d2c0-4d0c-a9d6-4a58e00ee747 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:51.447920 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbbe711-d2c0-4d0c-a9d6-4a58e00ee747 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:51.449909 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbbe711-d2c0-4d0c-a9d6-4a58e00ee747 service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:51.459482 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1327/2629] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:48:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:48:55.651581 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf3e23d-43bf-4535-a124-eb8678935c6a service cinder] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:55.660674 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf3e23d-43bf-4535-a124-eb8678935c6a service cinder] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:55.664404 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3604b68f-9866-4a85-b2e3-0bbeced9e98a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:55.665048 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604b68f-9866-4a85-b2e3-0bbeced9e98a service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:55.665337 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604b68f-9866-4a85-b2e3-0bbeced9e98a service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:55.665604 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604b68f-9866-4a85-b2e3-0bbeced9e98a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:55.728676 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3604b68f-9866-4a85-b2e3-0bbeced9e98a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:55.730769 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3604b68f-9866-4a85-b2e3-0bbeced9e98a service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:55.740654 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1303/2630] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:48:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:48:56.427936 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6daaa78-131d-4ac4-9f43-0dbaf28d38e4 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:56.428465 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6daaa78-131d-4ac4-9f43-0dbaf28d38e4 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:56.428572 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6daaa78-131d-4ac4-9f43-0dbaf28d38e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:56.449822 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-c6daaa78-131d-4ac4-9f43-0dbaf28d38e4 None None] MFA Rules not processed for user `bf905723382e4372ae63d77a376982ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:48:56.451014 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-c6daaa78-131d-4ac4-9f43-0dbaf28d38e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:56.558070 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1328/2631] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:48:56 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:48:58.162392 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3604b68f-9866-4a85-b2e3-0bbeced9e98a service cinder] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:58.164576 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-3604b68f-9866-4a85-b2e3-0bbeced9e98a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:58.212908 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6daaa78-131d-4ac4-9f43-0dbaf28d38e4 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:58.263238 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3604b68f-9866-4a85-b2e3-0bbeced9e98a service cinder] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:58.269811 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66964826-038d-4c08-a528-8ae470c0b00e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39cc268bad604dd08594e3b39c7ecf5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:58.270740 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66964826-038d-4c08-a528-8ae470c0b00e service placement] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:58.271322 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66964826-038d-4c08-a528-8ae470c0b00e service placement] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:58.271681 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66964826-038d-4c08-a528-8ae470c0b00e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:58.273754 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-66964826-038d-4c08-a528-8ae470c0b00e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:58.290117 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66964826-038d-4c08-a528-8ae470c0b00e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:58.292155 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66964826-038d-4c08-a528-8ae470c0b00e service placement] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:58.305087 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1304/2632] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:48:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:48:58.316244 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6daaa78-131d-4ac4-9f43-0dbaf28d38e4 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:58.322735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31318cdd-4840-4282-8e1f-8e5c2a439c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:58.325079 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31318cdd-4840-4282-8e1f-8e5c2a439c26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:58.325370 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31318cdd-4840-4282-8e1f-8e5c2a439c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:58.326785 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31318cdd-4840-4282-8e1f-8e5c2a439c26 admin admin] PATH_INFO: `/v3/users/9bb840d3901f459daf30082af46d3841` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:58.329770 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31318cdd-4840-4282-8e1f-8e5c2a439c26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bb840d3901f459daf30082af46d3841)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:58.332900 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31318cdd-4840-4282-8e1f-8e5c2a439c26 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:58.406871 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:48:58.412609 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-31318cdd-4840-4282-8e1f-8e5c2a439c26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bb840d3901f459daf30082af46d3841'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:58.433600 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-31318cdd-4840-4282-8e1f-8e5c2a439c26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bb840d3901f459daf30082af46d3841'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:58.443098 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-31318cdd-4840-4282-8e1f-8e5c2a439c26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bb840d3901f459daf30082af46d3841'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:58.443963 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-31318cdd-4840-4282-8e1f-8e5c2a439c26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bb840d3901f459daf30082af46d3841'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:58.445400 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-31318cdd-4840-4282-8e1f-8e5c2a439c26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bb840d3901f459daf30082af46d3841'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:58.474318 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1329/2633] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:48:58 2022] DELETE /identity/v3/users/9bb840d3901f459daf30082af46d3841 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:48:58.479504 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c2f975-8870-496e-bdfc-6ec9c5c0a40e None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:58.480531 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c2f975-8870-496e-bdfc-6ec9c5c0a40e None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:58.480937 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c2f975-8870-496e-bdfc-6ec9c5c0a40e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:58.505287 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-33c2f975-8870-496e-bdfc-6ec9c5c0a40e None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:48:58.506495 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-33c2f975-8870-496e-bdfc-6ec9c5c0a40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:58.526524 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:48:58.612650 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1305/2634] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:48:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:48:58.623535 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31318cdd-4840-4282-8e1f-8e5c2a439c26 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:58.624995 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-31318cdd-4840-4282-8e1f-8e5c2a439c26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:58.696613 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31318cdd-4840-4282-8e1f-8e5c2a439c26 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:58.701338 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec898da-363e-49e7-a849-4ab196f6445d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:58.702026 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec898da-363e-49e7-a849-4ab196f6445d service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:58.702315 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec898da-363e-49e7-a849-4ab196f6445d service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:58.702592 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec898da-363e-49e7-a849-4ab196f6445d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:58.713658 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec898da-363e-49e7-a849-4ab196f6445d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:58.716064 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec898da-363e-49e7-a849-4ab196f6445d service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:58.726096 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1330/2635] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:48:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:48:59.051600 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c2f975-8870-496e-bdfc-6ec9c5c0a40e None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:59.055074 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-33c2f975-8870-496e-bdfc-6ec9c5c0a40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:59.073237 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c2f975-8870-496e-bdfc-6ec9c5c0a40e None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:59.079007 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8983aa-4f22-4138-890b-0250f52eb4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:59.079800 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8983aa-4f22-4138-890b-0250f52eb4fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:59.080125 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8983aa-4f22-4138-890b-0250f52eb4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:59.080713 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8983aa-4f22-4138-890b-0250f52eb4fb admin admin] PATH_INFO: `/v3/projects/e0e5a32a8f674a0aa5aa91bb83b14233` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:59.082286 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8983aa-4f22-4138-890b-0250f52eb4fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0e5a32a8f674a0aa5aa91bb83b14233)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:59.084572 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8983aa-4f22-4138-890b-0250f52eb4fb admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:59.176143 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-4a8983aa-4f22-4138-890b-0250f52eb4fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0e5a32a8f674a0aa5aa91bb83b14233', 'request_id': 'req-4a8983aa-4f22-4138-890b-0250f52eb4fb'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:59.183547 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-4a8983aa-4f22-4138-890b-0250f52eb4fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0e5a32a8f674a0aa5aa91bb83b14233', 'request_id': 'req-4a8983aa-4f22-4138-890b-0250f52eb4fb'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:59.195518 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-4a8983aa-4f22-4138-890b-0250f52eb4fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0e5a32a8f674a0aa5aa91bb83b14233', 'request_id': 'req-4a8983aa-4f22-4138-890b-0250f52eb4fb'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:59.216016 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-4a8983aa-4f22-4138-890b-0250f52eb4fb admin admin] The token cache is being invalidate because project e0e5a32a8f674a0aa5aa91bb83b14233 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:48:59.216344 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-4a8983aa-4f22-4138-890b-0250f52eb4fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:59.217273 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-4a8983aa-4f22-4138-890b-0250f52eb4fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:59.219215 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1306/2636] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:48:59 2022] DELETE /identity/v3/projects/e0e5a32a8f674a0aa5aa91bb83b14233 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:48:59.223152 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec898da-363e-49e7-a849-4ab196f6445d service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:59.234629 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec898da-363e-49e7-a849-4ab196f6445d service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:59.328268 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c35bff-7fc0-4a3b-9e47-9ea5f50ebfbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:59.329282 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c35bff-7fc0-4a3b-9e47-9ea5f50ebfbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:59.329673 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c35bff-7fc0-4a3b-9e47-9ea5f50ebfbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:59.330202 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c35bff-7fc0-4a3b-9e47-9ea5f50ebfbe admin admin] PATH_INFO: `/v3/users/d44daccb967e47b1a5b15b5f574b38ac` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:59.331532 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c35bff-7fc0-4a3b-9e47-9ea5f50ebfbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=d44daccb967e47b1a5b15b5f574b38ac)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:59.333633 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c35bff-7fc0-4a3b-9e47-9ea5f50ebfbe admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:59.397381 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-66c35bff-7fc0-4a3b-9e47-9ea5f50ebfbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd44daccb967e47b1a5b15b5f574b38ac'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:59.410772 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-66c35bff-7fc0-4a3b-9e47-9ea5f50ebfbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd44daccb967e47b1a5b15b5f574b38ac'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:59.412708 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:48:59.419879 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-66c35bff-7fc0-4a3b-9e47-9ea5f50ebfbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd44daccb967e47b1a5b15b5f574b38ac'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:59.421482 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-66c35bff-7fc0-4a3b-9e47-9ea5f50ebfbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd44daccb967e47b1a5b15b5f574b38ac'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:59.422470 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-66c35bff-7fc0-4a3b-9e47-9ea5f50ebfbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd44daccb967e47b1a5b15b5f574b38ac'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:59.436599 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1331/2637] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:48:59 2022] DELETE /identity/v3/users/d44daccb967e47b1a5b15b5f574b38ac => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:48:59.772627 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8983aa-4f22-4138-890b-0250f52eb4fb admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:59.775026 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-4a8983aa-4f22-4138-890b-0250f52eb4fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:59.805858 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8983aa-4f22-4138-890b-0250f52eb4fb admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:59.895252 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad24707f-63de-4cbb-872b-e1cc8666baa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:59.896385 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24707f-63de-4cbb-872b-e1cc8666baa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:59.896676 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24707f-63de-4cbb-872b-e1cc8666baa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:59.896959 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24707f-63de-4cbb-872b-e1cc8666baa4 admin admin] PATH_INFO: `/v3/projects/5b2f50f418e14da68a6ed374ca5d8e78` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:59.899236 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad24707f-63de-4cbb-872b-e1cc8666baa4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b2f50f418e14da68a6ed374ca5d8e78)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:59.901206 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad24707f-63de-4cbb-872b-e1cc8666baa4 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:59.988868 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:48:59.994559 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-ad24707f-63de-4cbb-872b-e1cc8666baa4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b2f50f418e14da68a6ed374ca5d8e78', 'request_id': 'req-ad24707f-63de-4cbb-872b-e1cc8666baa4'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:00.009830 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-ad24707f-63de-4cbb-872b-e1cc8666baa4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b2f50f418e14da68a6ed374ca5d8e78', 'request_id': 'req-ad24707f-63de-4cbb-872b-e1cc8666baa4'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:00.021785 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-ad24707f-63de-4cbb-872b-e1cc8666baa4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b2f50f418e14da68a6ed374ca5d8e78', 'request_id': 'req-ad24707f-63de-4cbb-872b-e1cc8666baa4'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:00.043127 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-ad24707f-63de-4cbb-872b-e1cc8666baa4 admin admin] The token cache is being invalidate because project 5b2f50f418e14da68a6ed374ca5d8e78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:49:00.043602 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-ad24707f-63de-4cbb-872b-e1cc8666baa4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:00.044550 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-ad24707f-63de-4cbb-872b-e1cc8666baa4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:00.047654 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1307/2638] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:48:59 2022] DELETE /identity/v3/projects/5b2f50f418e14da68a6ed374ca5d8e78 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:49:00.052789 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c35bff-7fc0-4a3b-9e47-9ea5f50ebfbe admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:00.056178 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-66c35bff-7fc0-4a3b-9e47-9ea5f50ebfbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:49:00.074904 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c35bff-7fc0-4a3b-9e47-9ea5f50ebfbe admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:00.241484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1ff8d1-e041-4af8-bf9a-cf25f41b21ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:00.250094 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ff8d1-e041-4af8-bf9a-cf25f41b21ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:00.250399 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ff8d1-e041-4af8-bf9a-cf25f41b21ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:00.250694 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ff8d1-e041-4af8-bf9a-cf25f41b21ee admin admin] PATH_INFO: `/v3/users/06177ec4673f461d8a8d7baf2f611ba7` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:00.252276 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1ff8d1-e041-4af8-bf9a-cf25f41b21ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=06177ec4673f461d8a8d7baf2f611ba7)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:00.254892 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1ff8d1-e041-4af8-bf9a-cf25f41b21ee admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:00.320712 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0a1ff8d1-e041-4af8-bf9a-cf25f41b21ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06177ec4673f461d8a8d7baf2f611ba7'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:00.336546 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0a1ff8d1-e041-4af8-bf9a-cf25f41b21ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06177ec4673f461d8a8d7baf2f611ba7'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:00.345627 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0a1ff8d1-e041-4af8-bf9a-cf25f41b21ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06177ec4673f461d8a8d7baf2f611ba7'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:00.346430 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0a1ff8d1-e041-4af8-bf9a-cf25f41b21ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06177ec4673f461d8a8d7baf2f611ba7'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:00.347066 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0a1ff8d1-e041-4af8-bf9a-cf25f41b21ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06177ec4673f461d8a8d7baf2f611ba7'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:00.364394 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1332/2639] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:49:00 2022] DELETE /identity/v3/users/06177ec4673f461d8a8d7baf2f611ba7 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:49:00.882748 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad24707f-63de-4cbb-872b-e1cc8666baa4 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:00.884233 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-ad24707f-63de-4cbb-872b-e1cc8666baa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:49:00.914650 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad24707f-63de-4cbb-872b-e1cc8666baa4 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:01.008336 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f87aab8-97e9-4237-9dd6-d35e54511d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:01.009854 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f87aab8-97e9-4237-9dd6-d35e54511d93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:01.009854 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f87aab8-97e9-4237-9dd6-d35e54511d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:01.010279 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f87aab8-97e9-4237-9dd6-d35e54511d93 admin admin] PATH_INFO: `/v3/projects/640091cda2bd4938a1cfa42c70f1c3b8` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:01.012721 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f87aab8-97e9-4237-9dd6-d35e54511d93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=640091cda2bd4938a1cfa42c70f1c3b8)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:01.014743 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f87aab8-97e9-4237-9dd6-d35e54511d93 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:01.106981 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7f87aab8-97e9-4237-9dd6-d35e54511d93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '640091cda2bd4938a1cfa42c70f1c3b8', 'request_id': 'req-7f87aab8-97e9-4237-9dd6-d35e54511d93'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:01.117994 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7f87aab8-97e9-4237-9dd6-d35e54511d93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '640091cda2bd4938a1cfa42c70f1c3b8', 'request_id': 'req-7f87aab8-97e9-4237-9dd6-d35e54511d93'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:01.129298 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7f87aab8-97e9-4237-9dd6-d35e54511d93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '640091cda2bd4938a1cfa42c70f1c3b8', 'request_id': 'req-7f87aab8-97e9-4237-9dd6-d35e54511d93'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:01.153322 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7f87aab8-97e9-4237-9dd6-d35e54511d93 admin admin] The token cache is being invalidate because project 640091cda2bd4938a1cfa42c70f1c3b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:49:01.153979 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7f87aab8-97e9-4237-9dd6-d35e54511d93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:01.155201 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7f87aab8-97e9-4237-9dd6-d35e54511d93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:01.161889 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1308/2640] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:49:00 2022] DELETE /identity/v3/projects/640091cda2bd4938a1cfa42c70f1c3b8 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:49:01.167081 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f544d-aa9c-4b32-8d01-4014fb20dc53 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:01.167995 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f544d-aa9c-4b32-8d01-4014fb20dc53 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:01.168983 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f544d-aa9c-4b32-8d01-4014fb20dc53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:01.193078 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-d38f544d-aa9c-4b32-8d01-4014fb20dc53 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:49:01.194774 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-d38f544d-aa9c-4b32-8d01-4014fb20dc53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:49:01.298962 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1333/2641] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:49:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:49:01.309184 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efaa459-d29b-4fc7-951d-bc1781b46c8c None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:01.309820 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efaa459-d29b-4fc7-951d-bc1781b46c8c None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:01.310223 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efaa459-d29b-4fc7-951d-bc1781b46c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:01.343503 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-3efaa459-d29b-4fc7-951d-bc1781b46c8c None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:49:01.344913 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-3efaa459-d29b-4fc7-951d-bc1781b46c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:49:01.360703 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1309/2642] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:49:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:49:01.366077 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38f544d-aa9c-4b32-8d01-4014fb20dc53 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:01.372635 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:49:01.381556 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38f544d-aa9c-4b32-8d01-4014fb20dc53 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:01.397755 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fd4a2a-4d74-4866-9d84-241c68caf05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:01.397755 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd4a2a-4d74-4866-9d84-241c68caf05d admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:01.397755 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd4a2a-4d74-4866-9d84-241c68caf05d admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:01.398167 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd4a2a-4d74-4866-9d84-241c68caf05d admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:01.399318 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fd4a2a-4d74-4866-9d84-241c68caf05d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:49:01.399826 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fd4a2a-4d74-4866-9d84-241c68caf05d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:01.403132 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fd4a2a-4d74-4866-9d84-241c68caf05d admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:01.418753 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1334/2643] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:49:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:49:01.423348 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efaa459-d29b-4fc7-951d-bc1781b46c8c None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:01.434219 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efaa459-d29b-4fc7-951d-bc1781b46c8c None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:01.439943 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d0f32c-8102-4627-a63a-54dfbc9f7a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:01.440635 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d0f32c-8102-4627-a63a-54dfbc9f7a3b admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:01.440941 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d0f32c-8102-4627-a63a-54dfbc9f7a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:01.441197 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d0f32c-8102-4627-a63a-54dfbc9f7a3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:01.441979 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d0f32c-8102-4627-a63a-54dfbc9f7a3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:01.444116 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d0f32c-8102-4627-a63a-54dfbc9f7a3b admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:01.485567 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1310/2644] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:49:01 2022] POST /identity/v3/projects => generated 363 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:49:01.486946 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fd4a2a-4d74-4866-9d84-241c68caf05d admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:01.504541 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fd4a2a-4d74-4866-9d84-241c68caf05d admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:01.576529 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fbb321-caa1-4aaf-a094-540d9c04ecac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:01.577219 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbb321-caa1-4aaf-a094-540d9c04ecac admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:01.577505 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbb321-caa1-4aaf-a094-540d9c04ecac admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:01.577833 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbb321-caa1-4aaf-a094-540d9c04ecac admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:01.578490 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fbb321-caa1-4aaf-a094-540d9c04ecac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:01.580499 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fbb321-caa1-4aaf-a094-540d9c04ecac admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:01.604870 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1335/2645] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:49:01 2022] POST /identity/v3/users => generated 282 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:49:01.606416 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d0f32c-8102-4627-a63a-54dfbc9f7a3b admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:01.618582 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d0f32c-8102-4627-a63a-54dfbc9f7a3b admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:01.624269 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b664f42-11f0-4639-8a79-69d6e4a5d038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:01.624972 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b664f42-11f0-4639-8a79-69d6e4a5d038 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:01.625255 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b664f42-11f0-4639-8a79-69d6e4a5d038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:01.625531 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b664f42-11f0-4639-8a79-69d6e4a5d038 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:01.626307 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b664f42-11f0-4639-8a79-69d6e4a5d038 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:01.627972 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b664f42-11f0-4639-8a79-69d6e4a5d038 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:01.645225 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1311/2646] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:49:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:49:01.646418 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fbb321-caa1-4aaf-a094-540d9c04ecac admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:01.657850 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fbb321-caa1-4aaf-a094-540d9c04ecac admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:01.662845 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe123d5-f079-4115-919b-30e432b32378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:01.663519 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe123d5-f079-4115-919b-30e432b32378 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:01.663799 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe123d5-f079-4115-919b-30e432b32378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:01.664076 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe123d5-f079-4115-919b-30e432b32378 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:01.664811 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe123d5-f079-4115-919b-30e432b32378 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:01.666497 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe123d5-f079-4115-919b-30e432b32378 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:01.678651 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1336/2647] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:49:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:49:01.683597 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b664f42-11f0-4639-8a79-69d6e4a5d038 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:01.694666 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b664f42-11f0-4639-8a79-69d6e4a5d038 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:01.700105 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7895312d-1f1d-4143-85d5-08c4e25ae523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:01.701134 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895312d-1f1d-4143-85d5-08c4e25ae523 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:01.701345 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895312d-1f1d-4143-85d5-08c4e25ae523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:01.701658 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895312d-1f1d-4143-85d5-08c4e25ae523 admin admin] PATH_INFO: `/v3/projects/d0f938ccd5fd4617b11dbab3f7a91b39/users/6527113eb22e47db8138ac8c2f9c9437/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:01.720915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7895312d-1f1d-4143-85d5-08c4e25ae523 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0f938ccd5fd4617b11dbab3f7a91b39, user_id=6527113eb22e47db8138ac8c2f9c9437, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:01.726216 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7895312d-1f1d-4143-85d5-08c4e25ae523 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:01.738631 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:49:01.744591 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1312/2648] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:49:01 2022] PUT /identity/v3/projects/d0f938ccd5fd4617b11dbab3f7a91b39/users/6527113eb22e47db8138ac8c2f9c9437/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:49:01.759108 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb4ab3f-348d-4c58-8270-9819176dfd59 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:01.759402 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb4ab3f-348d-4c58-8270-9819176dfd59 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:01.759687 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb4ab3f-348d-4c58-8270-9819176dfd59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:01.788683 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-1fb4ab3f-348d-4c58-8270-9819176dfd59 None None] MFA Rules not processed for user `6527113eb22e47db8138ac8c2f9c9437`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:49:01.789951 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-1fb4ab3f-348d-4c58-8270-9819176dfd59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:49:01.860067 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1337/2649] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:49:01 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:49:01.864625 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7895312d-1f1d-4143-85d5-08c4e25ae523 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:01.930339 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7895312d-1f1d-4143-85d5-08c4e25ae523 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:01.936142 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8041fdd-f9fb-49ec-8908-20252f6b2dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:01.936806 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8041fdd-f9fb-49ec-8908-20252f6b2dfc admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:01.937089 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8041fdd-f9fb-49ec-8908-20252f6b2dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:01.937367 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8041fdd-f9fb-49ec-8908-20252f6b2dfc admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:01.938092 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8041fdd-f9fb-49ec-8908-20252f6b2dfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:01.940042 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8041fdd-f9fb-49ec-8908-20252f6b2dfc admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:01.969820 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1313/2650] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:49:01 2022] POST /identity/v3/projects => generated 367 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:49:01.974254 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb4ab3f-348d-4c58-8270-9819176dfd59 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:01.987554 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb4ab3f-348d-4c58-8270-9819176dfd59 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:02.047212 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:49:02.057187 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2493aa98-a008-49f5-8ae0-5153fa6ca07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:02.059030 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2493aa98-a008-49f5-8ae0-5153fa6ca07b admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:02.059733 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2493aa98-a008-49f5-8ae0-5153fa6ca07b admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:02.060268 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2493aa98-a008-49f5-8ae0-5153fa6ca07b admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:02.061235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2493aa98-a008-49f5-8ae0-5153fa6ca07b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:02.063236 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2493aa98-a008-49f5-8ae0-5153fa6ca07b admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:02.088230 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1338/2651] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:49:01 2022] POST /identity/v3/users => generated 290 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:49:02.090363 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8041fdd-f9fb-49ec-8908-20252f6b2dfc admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:02.102224 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8041fdd-f9fb-49ec-8908-20252f6b2dfc admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:02.108046 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7264ff-7aa4-41db-a300-c2b1550ff227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:02.108818 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7264ff-7aa4-41db-a300-c2b1550ff227 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:02.109084 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7264ff-7aa4-41db-a300-c2b1550ff227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:02.109258 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7264ff-7aa4-41db-a300-c2b1550ff227 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:02.110027 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7264ff-7aa4-41db-a300-c2b1550ff227 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:02.111725 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7264ff-7aa4-41db-a300-c2b1550ff227 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:02.124596 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1314/2652] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:49:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:49:02.128884 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2493aa98-a008-49f5-8ae0-5153fa6ca07b admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:02.139421 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2493aa98-a008-49f5-8ae0-5153fa6ca07b admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:02.144247 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09634ac-1ce1-4612-9c97-b4661d9e6857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:02.145166 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09634ac-1ce1-4612-9c97-b4661d9e6857 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:02.145453 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09634ac-1ce1-4612-9c97-b4661d9e6857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:02.145755 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09634ac-1ce1-4612-9c97-b4661d9e6857 admin admin] PATH_INFO: `/v3/projects/777fb01509c7446197852bbe35d65055/users/d2f1e4611b084852a316dc8429569c7c/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:02.162564 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09634ac-1ce1-4612-9c97-b4661d9e6857 admin admin] RBAC: Authorizing `identity:create_grant(project_id=777fb01509c7446197852bbe35d65055, user_id=d2f1e4611b084852a316dc8429569c7c, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:02.165258 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09634ac-1ce1-4612-9c97-b4661d9e6857 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:02.181457 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1339/2653] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:49:02 2022] PUT /identity/v3/projects/777fb01509c7446197852bbe35d65055/users/d2f1e4611b084852a316dc8429569c7c/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:49:02.185426 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7264ff-7aa4-41db-a300-c2b1550ff227 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:02.195865 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7264ff-7aa4-41db-a300-c2b1550ff227 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:02.264537 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54ce902-6e1f-4ceb-96cd-746b9e907b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:02.265245 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ce902-6e1f-4ceb-96cd-746b9e907b62 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:02.265527 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ce902-6e1f-4ceb-96cd-746b9e907b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:02.265837 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ce902-6e1f-4ceb-96cd-746b9e907b62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:02.266582 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54ce902-6e1f-4ceb-96cd-746b9e907b62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:02.268286 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54ce902-6e1f-4ceb-96cd-746b9e907b62 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:02.283520 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1315/2654] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:49:02 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:49:02.284748 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09634ac-1ce1-4612-9c97-b4661d9e6857 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:02.296084 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09634ac-1ce1-4612-9c97-b4661d9e6857 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:02.300878 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044d854e-2fc3-4579-91a2-4cf9e096e28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:02.301792 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d854e-2fc3-4579-91a2-4cf9e096e28c admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:02.302074 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d854e-2fc3-4579-91a2-4cf9e096e28c admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:02.302375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d854e-2fc3-4579-91a2-4cf9e096e28c admin admin] PATH_INFO: `/v3/projects/777fb01509c7446197852bbe35d65055/users/d2f1e4611b084852a316dc8429569c7c/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:02.304539 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044d854e-2fc3-4579-91a2-4cf9e096e28c admin admin] RBAC: Authorizing `identity:create_grant(project_id=777fb01509c7446197852bbe35d65055, user_id=d2f1e4611b084852a316dc8429569c7c, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:02.307138 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044d854e-2fc3-4579-91a2-4cf9e096e28c admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:02.319192 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1340/2655] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:49:02 2022] PUT /identity/v3/projects/777fb01509c7446197852bbe35d65055/users/d2f1e4611b084852a316dc8429569c7c/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:49:02.330226 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f613121a-ecb4-458d-bded-87fc91f616e9 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:02.330630 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f613121a-ecb4-458d-bded-87fc91f616e9 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:02.330915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f613121a-ecb4-458d-bded-87fc91f616e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:02.367017 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-f613121a-ecb4-458d-bded-87fc91f616e9 None None] MFA Rules not processed for user `d2f1e4611b084852a316dc8429569c7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:49:02.368528 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-f613121a-ecb4-458d-bded-87fc91f616e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:49:02.437332 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:49:02.455995 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1316/2656] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:49:02 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:49:02.463486 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044d854e-2fc3-4579-91a2-4cf9e096e28c admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:02.466299 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-044d854e-2fc3-4579-91a2-4cf9e096e28c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:49:02.528936 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044d854e-2fc3-4579-91a2-4cf9e096e28c admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:02.532804 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7474e48-97d4-4324-b374-116d5d15921c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:02.533569 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7474e48-97d4-4324-b374-116d5d15921c service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:02.533894 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7474e48-97d4-4324-b374-116d5d15921c service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:02.534182 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7474e48-97d4-4324-b374-116d5d15921c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:02.545437 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7474e48-97d4-4324-b374-116d5d15921c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:02.546944 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7474e48-97d4-4324-b374-116d5d15921c service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:02.557432 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1341/2657] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:49:02 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:49:02.650327 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f613121a-ecb4-458d-bded-87fc91f616e9 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:02.664191 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f613121a-ecb4-458d-bded-87fc91f616e9 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:02.668823 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc832325-c6ad-4356-9af0-99f5c11f4bbf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:02.669581 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc832325-c6ad-4356-9af0-99f5c11f4bbf service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:02.669975 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc832325-c6ad-4356-9af0-99f5c11f4bbf service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:02.670254 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc832325-c6ad-4356-9af0-99f5c11f4bbf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:02.736307 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc832325-c6ad-4356-9af0-99f5c11f4bbf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:02.737889 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc832325-c6ad-4356-9af0-99f5c11f4bbf service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:02.749507 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1317/2658] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:49:02 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:49:07.492026 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7474e48-97d4-4324-b374-116d5d15921c service cinder] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:07.559852 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7474e48-97d4-4324-b374-116d5d15921c service cinder] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:07.564835 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1160e1f-115b-4f4b-ae5e-35fe0b52bb58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:07.565776 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1160e1f-115b-4f4b-ae5e-35fe0b52bb58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:07.566006 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1160e1f-115b-4f4b-ae5e-35fe0b52bb58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:07.566275 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1160e1f-115b-4f4b-ae5e-35fe0b52bb58 admin admin] PATH_INFO: `/v3/users/6527113eb22e47db8138ac8c2f9c9437` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:07.567493 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1160e1f-115b-4f4b-ae5e-35fe0b52bb58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6527113eb22e47db8138ac8c2f9c9437)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:07.569398 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1160e1f-115b-4f4b-ae5e-35fe0b52bb58 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:07.624846 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d1160e1f-115b-4f4b-ae5e-35fe0b52bb58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6527113eb22e47db8138ac8c2f9c9437'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:07.630349 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=87,keystone:INSERT=5,keystone:DELETE=8 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:49:07.639962 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d1160e1f-115b-4f4b-ae5e-35fe0b52bb58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6527113eb22e47db8138ac8c2f9c9437'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:07.648869 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d1160e1f-115b-4f4b-ae5e-35fe0b52bb58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6527113eb22e47db8138ac8c2f9c9437'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:07.649551 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d1160e1f-115b-4f4b-ae5e-35fe0b52bb58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6527113eb22e47db8138ac8c2f9c9437'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:07.650194 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d1160e1f-115b-4f4b-ae5e-35fe0b52bb58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6527113eb22e47db8138ac8c2f9c9437'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:07.664235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1342/2659] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:49:07 2022] DELETE /identity/v3/users/6527113eb22e47db8138ac8c2f9c9437 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:49:07.670286 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ebb9b-8690-43a0-aecf-30ec913bc6b8 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:07.670737 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ebb9b-8690-43a0-aecf-30ec913bc6b8 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:07.671127 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ebb9b-8690-43a0-aecf-30ec913bc6b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:07.689876 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-d78ebb9b-8690-43a0-aecf-30ec913bc6b8 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:49:07.691089 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d78ebb9b-8690-43a0-aecf-30ec913bc6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:49:07.750244 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1318/2660] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:49:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:49:07.760343 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1160e1f-115b-4f4b-ae5e-35fe0b52bb58 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:07.763334 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-d1160e1f-115b-4f4b-ae5e-35fe0b52bb58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:49:07.820778 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1160e1f-115b-4f4b-ae5e-35fe0b52bb58 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:07.825088 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8962d9cd-d281-46da-aadd-084f5976ed16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:07.825763 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8962d9cd-d281-46da-aadd-084f5976ed16 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:07.826054 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8962d9cd-d281-46da-aadd-084f5976ed16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:07.826337 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8962d9cd-d281-46da-aadd-084f5976ed16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:07.835721 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8962d9cd-d281-46da-aadd-084f5976ed16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:07.837154 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8962d9cd-d281-46da-aadd-084f5976ed16 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:07.847490 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1343/2661] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:49:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:49:08.125933 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78ebb9b-8690-43a0-aecf-30ec913bc6b8 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:08.127500 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d78ebb9b-8690-43a0-aecf-30ec913bc6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:49:08.140736 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78ebb9b-8690-43a0-aecf-30ec913bc6b8 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:08.145582 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92f7e87-c0c5-4453-bb9b-235dd680b78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:08.146500 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92f7e87-c0c5-4453-bb9b-235dd680b78f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:08.146788 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92f7e87-c0c5-4453-bb9b-235dd680b78f admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:08.147066 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92f7e87-c0c5-4453-bb9b-235dd680b78f admin admin] PATH_INFO: `/v3/projects/d0f938ccd5fd4617b11dbab3f7a91b39` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:08.148256 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92f7e87-c0c5-4453-bb9b-235dd680b78f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0f938ccd5fd4617b11dbab3f7a91b39)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:08.150203 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92f7e87-c0c5-4453-bb9b-235dd680b78f admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:08.200680 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f92f7e87-c0c5-4453-bb9b-235dd680b78f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0f938ccd5fd4617b11dbab3f7a91b39', 'request_id': 'req-f92f7e87-c0c5-4453-bb9b-235dd680b78f'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:08.206804 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f92f7e87-c0c5-4453-bb9b-235dd680b78f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0f938ccd5fd4617b11dbab3f7a91b39', 'request_id': 'req-f92f7e87-c0c5-4453-bb9b-235dd680b78f'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:08.212285 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f92f7e87-c0c5-4453-bb9b-235dd680b78f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0f938ccd5fd4617b11dbab3f7a91b39', 'request_id': 'req-f92f7e87-c0c5-4453-bb9b-235dd680b78f'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:08.225424 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f92f7e87-c0c5-4453-bb9b-235dd680b78f admin admin] The token cache is being invalidate because project d0f938ccd5fd4617b11dbab3f7a91b39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:49:08.225735 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f92f7e87-c0c5-4453-bb9b-235dd680b78f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:08.226349 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f92f7e87-c0c5-4453-bb9b-235dd680b78f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:08.227868 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1319/2662] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:49:08 2022] DELETE /identity/v3/projects/d0f938ccd5fd4617b11dbab3f7a91b39 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:49:08.231916 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8962d9cd-d281-46da-aadd-084f5976ed16 service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:08.241487 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8962d9cd-d281-46da-aadd-084f5976ed16 service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:08.299581 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fe031d-6f46-43c2-9102-be2e8f9721c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:08.300450 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe031d-6f46-43c2-9102-be2e8f9721c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:08.300737 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe031d-6f46-43c2-9102-be2e8f9721c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:08.301014 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe031d-6f46-43c2-9102-be2e8f9721c3 admin admin] PATH_INFO: `/v3/users/d2f1e4611b084852a316dc8429569c7c` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:08.302241 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fe031d-6f46-43c2-9102-be2e8f9721c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2f1e4611b084852a316dc8429569c7c)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:08.304162 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fe031d-6f46-43c2-9102-be2e8f9721c3 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:08.351516 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-e2fe031d-6f46-43c2-9102-be2e8f9721c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2f1e4611b084852a316dc8429569c7c'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:08.366220 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-e2fe031d-6f46-43c2-9102-be2e8f9721c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2f1e4611b084852a316dc8429569c7c'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:08.375249 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-e2fe031d-6f46-43c2-9102-be2e8f9721c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2f1e4611b084852a316dc8429569c7c'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:08.375914 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-e2fe031d-6f46-43c2-9102-be2e8f9721c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2f1e4611b084852a316dc8429569c7c'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:08.376515 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-e2fe031d-6f46-43c2-9102-be2e8f9721c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2f1e4611b084852a316dc8429569c7c'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:08.381738 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:49:08.392838 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1344/2663] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:49:08 2022] DELETE /identity/v3/users/d2f1e4611b084852a316dc8429569c7c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:49:08.672974 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92f7e87-c0c5-4453-bb9b-235dd680b78f admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:08.674517 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-f92f7e87-c0c5-4453-bb9b-235dd680b78f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:49:08.689018 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92f7e87-c0c5-4453-bb9b-235dd680b78f admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:08.704062 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:49:08.755884 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b756cdd7-8212-4c93-b88a-6b7c3ffcf69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:08.756864 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b756cdd7-8212-4c93-b88a-6b7c3ffcf69e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:08.757149 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b756cdd7-8212-4c93-b88a-6b7c3ffcf69e admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:08.757433 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b756cdd7-8212-4c93-b88a-6b7c3ffcf69e admin admin] PATH_INFO: `/v3/projects/777fb01509c7446197852bbe35d65055` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:08.758778 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b756cdd7-8212-4c93-b88a-6b7c3ffcf69e admin admin] RBAC: Authorizing `identity:delete_project(project_id=777fb01509c7446197852bbe35d65055)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:08.760749 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b756cdd7-8212-4c93-b88a-6b7c3ffcf69e admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:08.813639 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-b756cdd7-8212-4c93-b88a-6b7c3ffcf69e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '777fb01509c7446197852bbe35d65055', 'request_id': 'req-b756cdd7-8212-4c93-b88a-6b7c3ffcf69e'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:08.820374 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-b756cdd7-8212-4c93-b88a-6b7c3ffcf69e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '777fb01509c7446197852bbe35d65055', 'request_id': 'req-b756cdd7-8212-4c93-b88a-6b7c3ffcf69e'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:08.826398 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-b756cdd7-8212-4c93-b88a-6b7c3ffcf69e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '777fb01509c7446197852bbe35d65055', 'request_id': 'req-b756cdd7-8212-4c93-b88a-6b7c3ffcf69e'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:08.840858 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-b756cdd7-8212-4c93-b88a-6b7c3ffcf69e admin admin] The token cache is being invalidate because project 777fb01509c7446197852bbe35d65055 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:49:08.841150 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-b756cdd7-8212-4c93-b88a-6b7c3ffcf69e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:08.841857 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-b756cdd7-8212-4c93-b88a-6b7c3ffcf69e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:08.843443 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1320/2664] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:49:08 2022] DELETE /identity/v3/projects/777fb01509c7446197852bbe35d65055 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:49:08.848332 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e439dde-fbd3-4101-833c-e743cd478cd0 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:08.848823 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e439dde-fbd3-4101-833c-e743cd478cd0 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:08.848930 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e439dde-fbd3-4101-833c-e743cd478cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:08.869966 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-8e439dde-fbd3-4101-833c-e743cd478cd0 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:49:08.871309 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-8e439dde-fbd3-4101-833c-e743cd478cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:49:08.938125 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1345/2665] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:49:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:49:08.942452 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef577b12-b265-4bca-8369-3a437a53bc75 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:08.942748 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef577b12-b265-4bca-8369-3a437a53bc75 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:08.943032 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef577b12-b265-4bca-8369-3a437a53bc75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:08.961547 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-ef577b12-b265-4bca-8369-3a437a53bc75 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:49:08.962847 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-ef577b12-b265-4bca-8369-3a437a53bc75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:49:08.971980 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1321/2666] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:49:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:49:08.976208 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e439dde-fbd3-4101-833c-e743cd478cd0 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:08.988451 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e439dde-fbd3-4101-833c-e743cd478cd0 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:08.993185 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a92be1-915b-4aa4-80aa-5d4f6b05c9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:08.993888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a92be1-915b-4aa4-80aa-5d4f6b05c9ba admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:08.994171 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a92be1-915b-4aa4-80aa-5d4f6b05c9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:08.994448 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a92be1-915b-4aa4-80aa-5d4f6b05c9ba admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:08.995220 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a92be1-915b-4aa4-80aa-5d4f6b05c9ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:49:08.995614 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a92be1-915b-4aa4-80aa-5d4f6b05c9ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:08.997305 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a92be1-915b-4aa4-80aa-5d4f6b05c9ba admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:09.014101 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1346/2667] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:49:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:49:09.015379 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef577b12-b265-4bca-8369-3a437a53bc75 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:09.025137 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef577b12-b265-4bca-8369-3a437a53bc75 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:09.031210 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05005410-9151-4250-b739-573d26179de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:09.032023 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05005410-9151-4250-b739-573d26179de1 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:09.032377 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05005410-9151-4250-b739-573d26179de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:09.032722 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05005410-9151-4250-b739-573d26179de1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:09.033610 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05005410-9151-4250-b739-573d26179de1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:09.036043 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05005410-9151-4250-b739-573d26179de1 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:09.070228 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1322/2668] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:49:09 2022] POST /identity/v3/projects => generated 361 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:49:09.071558 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a92be1-915b-4aa4-80aa-5d4f6b05c9ba admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:09.082533 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a92be1-915b-4aa4-80aa-5d4f6b05c9ba admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:09.141648 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db55cc4-e725-45f4-8fad-de8120574924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:09.142350 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db55cc4-e725-45f4-8fad-de8120574924 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:09.142641 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db55cc4-e725-45f4-8fad-de8120574924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:09.142921 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db55cc4-e725-45f4-8fad-de8120574924 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:09.143622 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db55cc4-e725-45f4-8fad-de8120574924 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:09.145541 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db55cc4-e725-45f4-8fad-de8120574924 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:09.166546 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1347/2669] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:49:09 2022] POST /identity/v3/users => generated 281 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:49:09.170869 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05005410-9151-4250-b739-573d26179de1 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:09.181196 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05005410-9151-4250-b739-573d26179de1 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:09.186050 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdba9f1b-7bc8-4feb-ab33-41fc3920666f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:09.186707 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba9f1b-7bc8-4feb-ab33-41fc3920666f admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:09.186994 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba9f1b-7bc8-4feb-ab33-41fc3920666f admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:09.187277 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba9f1b-7bc8-4feb-ab33-41fc3920666f admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:09.187997 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdba9f1b-7bc8-4feb-ab33-41fc3920666f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:09.189602 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdba9f1b-7bc8-4feb-ab33-41fc3920666f admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:09.201165 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1323/2670] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:49:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:49:09.205410 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db55cc4-e725-45f4-8fad-de8120574924 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:09.216505 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db55cc4-e725-45f4-8fad-de8120574924 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:09.221233 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce0523-4197-42a5-a6a0-e7b949ef3c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:09.221929 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce0523-4197-42a5-a6a0-e7b949ef3c0a admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:09.222218 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce0523-4197-42a5-a6a0-e7b949ef3c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:09.222498 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce0523-4197-42a5-a6a0-e7b949ef3c0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:09.223225 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ce0523-4197-42a5-a6a0-e7b949ef3c0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:09.224810 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ce0523-4197-42a5-a6a0-e7b949ef3c0a admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:09.235301 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1348/2671] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:49:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:49:09.239624 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdba9f1b-7bc8-4feb-ab33-41fc3920666f admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:09.249916 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdba9f1b-7bc8-4feb-ab33-41fc3920666f admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:09.256928 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5c0cd6-877b-4e73-a944-2310dcfa5c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:09.257961 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5c0cd6-877b-4e73-a944-2310dcfa5c7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:09.258279 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5c0cd6-877b-4e73-a944-2310dcfa5c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:09.258562 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5c0cd6-877b-4e73-a944-2310dcfa5c7e admin admin] PATH_INFO: `/v3/projects/5a53c17b0c0641d3a12a924ab5c3fe7b/users/bc98e00cce244977be729870764ec786/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:09.275220 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5c0cd6-877b-4e73-a944-2310dcfa5c7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a53c17b0c0641d3a12a924ab5c3fe7b, user_id=bc98e00cce244977be729870764ec786, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:09.278549 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5c0cd6-877b-4e73-a944-2310dcfa5c7e admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:09.293015 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1324/2672] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:49:09 2022] PUT /identity/v3/projects/5a53c17b0c0641d3a12a924ab5c3fe7b/users/bc98e00cce244977be729870764ec786/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:49:09.304231 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a76dcc-d5b0-4e78-8e74-877717b95a83 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:09.304521 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a76dcc-d5b0-4e78-8e74-877717b95a83 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:09.305007 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a76dcc-d5b0-4e78-8e74-877717b95a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:09.333620 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-d0a76dcc-d5b0-4e78-8e74-877717b95a83 None None] MFA Rules not processed for user `bc98e00cce244977be729870764ec786`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:49:09.334881 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-d0a76dcc-d5b0-4e78-8e74-877717b95a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:49:09.393909 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:49:09.416125 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1349/2673] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:49:09 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:49:09.451235 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5c0cd6-877b-4e73-a944-2310dcfa5c7e admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:09.453154 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-0f5c0cd6-877b-4e73-a944-2310dcfa5c7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:49:09.511790 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:49:09.525217 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5c0cd6-877b-4e73-a944-2310dcfa5c7e admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:09.530676 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dab04aa-4c06-4402-8242-6e4dea2dd1b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:09.531338 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab04aa-4c06-4402-8242-6e4dea2dd1b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:09.531627 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab04aa-4c06-4402-8242-6e4dea2dd1b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:09.531905 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab04aa-4c06-4402-8242-6e4dea2dd1b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:09.541211 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dab04aa-4c06-4402-8242-6e4dea2dd1b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:09.543367 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dab04aa-4c06-4402-8242-6e4dea2dd1b5 service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:09.553514 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1325/2674] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:49:09 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:49:09.568308 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a76dcc-d5b0-4e78-8e74-877717b95a83 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:09.648043 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a76dcc-d5b0-4e78-8e74-877717b95a83 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:09.653926 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d72e343-48ca-4b7c-8180-1f96a5c8c9a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:09.654766 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72e343-48ca-4b7c-8180-1f96a5c8c9a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:09.655025 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72e343-48ca-4b7c-8180-1f96a5c8c9a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:09.655299 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72e343-48ca-4b7c-8180-1f96a5c8c9a1 admin admin] PATH_INFO: `/v3/users/bc98e00cce244977be729870764ec786` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:09.656551 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d72e343-48ca-4b7c-8180-1f96a5c8c9a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc98e00cce244977be729870764ec786)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:09.658544 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d72e343-48ca-4b7c-8180-1f96a5c8c9a1 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:09.706682 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-1d72e343-48ca-4b7c-8180-1f96a5c8c9a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc98e00cce244977be729870764ec786'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:09.719780 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-1d72e343-48ca-4b7c-8180-1f96a5c8c9a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc98e00cce244977be729870764ec786'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:09.727212 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-1d72e343-48ca-4b7c-8180-1f96a5c8c9a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc98e00cce244977be729870764ec786'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:09.727869 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-1d72e343-48ca-4b7c-8180-1f96a5c8c9a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc98e00cce244977be729870764ec786'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:09.728479 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-1d72e343-48ca-4b7c-8180-1f96a5c8c9a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc98e00cce244977be729870764ec786'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:09.741465 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1350/2675] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:49:09 2022] DELETE /identity/v3/users/bc98e00cce244977be729870764ec786 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:49:09.746041 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291267bd-0626-47f5-a83b-2313ee00d66a None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:09.746558 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291267bd-0626-47f5-a83b-2313ee00d66a None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:09.746558 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291267bd-0626-47f5-a83b-2313ee00d66a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:09.764317 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-291267bd-0626-47f5-a83b-2313ee00d66a None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:49:09.765541 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-291267bd-0626-47f5-a83b-2313ee00d66a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:49:09.828625 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1326/2676] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:49:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:49:09.838364 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d72e343-48ca-4b7c-8180-1f96a5c8c9a1 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:09.840595 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-1d72e343-48ca-4b7c-8180-1f96a5c8c9a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:49:09.921817 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d72e343-48ca-4b7c-8180-1f96a5c8c9a1 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:09.926643 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c8f035-3a66-4bc0-9ba1-4d215a2c020b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:09.927377 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8f035-3a66-4bc0-9ba1-4d215a2c020b service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:09.927661 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8f035-3a66-4bc0-9ba1-4d215a2c020b service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:09.927935 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8f035-3a66-4bc0-9ba1-4d215a2c020b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:09.938238 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c8f035-3a66-4bc0-9ba1-4d215a2c020b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:09.939707 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c8f035-3a66-4bc0-9ba1-4d215a2c020b service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:09.951481 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1351/2677] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:49:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:49:10.350803 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291267bd-0626-47f5-a83b-2313ee00d66a None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:10.352261 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-291267bd-0626-47f5-a83b-2313ee00d66a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:49:10.365457 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291267bd-0626-47f5-a83b-2313ee00d66a None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:10.379890 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d418b31-1261-4aed-8944-e121c4294fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:10.381565 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d418b31-1261-4aed-8944-e121c4294fc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:10.382115 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d418b31-1261-4aed-8944-e121c4294fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:10.383388 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d418b31-1261-4aed-8944-e121c4294fc2 admin admin] PATH_INFO: `/v3/projects/5a53c17b0c0641d3a12a924ab5c3fe7b` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:10.385170 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d418b31-1261-4aed-8944-e121c4294fc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a53c17b0c0641d3a12a924ab5c3fe7b)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:10.390984 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d418b31-1261-4aed-8944-e121c4294fc2 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:10.443961 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-1d418b31-1261-4aed-8944-e121c4294fc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a53c17b0c0641d3a12a924ab5c3fe7b', 'request_id': 'req-1d418b31-1261-4aed-8944-e121c4294fc2'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:10.453994 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-1d418b31-1261-4aed-8944-e121c4294fc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a53c17b0c0641d3a12a924ab5c3fe7b', 'request_id': 'req-1d418b31-1261-4aed-8944-e121c4294fc2'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:10.460243 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-1d418b31-1261-4aed-8944-e121c4294fc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a53c17b0c0641d3a12a924ab5c3fe7b', 'request_id': 'req-1d418b31-1261-4aed-8944-e121c4294fc2'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:10.473596 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-1d418b31-1261-4aed-8944-e121c4294fc2 admin admin] The token cache is being invalidate because project 5a53c17b0c0641d3a12a924ab5c3fe7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:49:10.473910 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-1d418b31-1261-4aed-8944-e121c4294fc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:10.474571 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-1d418b31-1261-4aed-8944-e121c4294fc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:10.476142 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1327/2678] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:49:10 2022] DELETE /identity/v3/projects/5a53c17b0c0641d3a12a924ab5c3fe7b => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:49:10.488120 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3658e8-4087-455b-99c4-b107c086c167 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:10.488440 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3658e8-4087-455b-99c4-b107c086c167 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:10.488746 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3658e8-4087-455b-99c4-b107c086c167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:10.508481 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-5b3658e8-4087-455b-99c4-b107c086c167 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:49:10.510854 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-5b3658e8-4087-455b-99c4-b107c086c167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:49:10.517645 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:49:10.589998 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1352/2679] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:49:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:49:10.597936 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b864a-293f-46f1-ab9f-2877aeab11fd None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:10.598224 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b864a-293f-46f1-ab9f-2877aeab11fd None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:10.598508 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b864a-293f-46f1-ab9f-2877aeab11fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:10.618300 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-5b8b864a-293f-46f1-ab9f-2877aeab11fd None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:49:10.619811 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-5b8b864a-293f-46f1-ab9f-2877aeab11fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:49:10.629397 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1328/2680] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:49:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:49:10.633787 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3658e8-4087-455b-99c4-b107c086c167 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:10.646684 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3658e8-4087-455b-99c4-b107c086c167 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:10.651801 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3303fe21-5d56-43f5-a5ef-329cf1729f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:10.652479 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3303fe21-5d56-43f5-a5ef-329cf1729f86 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:10.652765 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3303fe21-5d56-43f5-a5ef-329cf1729f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:10.653042 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3303fe21-5d56-43f5-a5ef-329cf1729f86 admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:10.653806 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3303fe21-5d56-43f5-a5ef-329cf1729f86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:49:10.654118 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3303fe21-5d56-43f5-a5ef-329cf1729f86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:10.655725 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3303fe21-5d56-43f5-a5ef-329cf1729f86 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:10.671410 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1353/2681] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:49:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:49:10.675519 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b864a-293f-46f1-ab9f-2877aeab11fd None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:10.686089 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b864a-293f-46f1-ab9f-2877aeab11fd None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:10.691189 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b11342-f865-4829-bb7e-1d9b49fe92e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:10.691883 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b11342-f865-4829-bb7e-1d9b49fe92e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:10.692168 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b11342-f865-4829-bb7e-1d9b49fe92e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:10.692445 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b11342-f865-4829-bb7e-1d9b49fe92e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:10.693137 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b11342-f865-4829-bb7e-1d9b49fe92e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:10.695147 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b11342-f865-4829-bb7e-1d9b49fe92e7 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:10.724810 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1329/2682] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:49:10 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:49:10.729159 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3303fe21-5d56-43f5-a5ef-329cf1729f86 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:10.739648 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3303fe21-5d56-43f5-a5ef-329cf1729f86 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:10.810938 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fea22fd-267e-4b55-8996-1bf1e4599553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:10.812005 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea22fd-267e-4b55-8996-1bf1e4599553 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:10.812474 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea22fd-267e-4b55-8996-1bf1e4599553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:10.812936 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea22fd-267e-4b55-8996-1bf1e4599553 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:10.814220 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fea22fd-267e-4b55-8996-1bf1e4599553 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:10.817504 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fea22fd-267e-4b55-8996-1bf1e4599553 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:10.845106 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1354/2683] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:49:10 2022] POST /identity/v3/users => generated 288 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:49:10.849513 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b11342-f865-4829-bb7e-1d9b49fe92e7 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:10.862172 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b11342-f865-4829-bb7e-1d9b49fe92e7 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:10.867689 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa0212-2f09-4e37-bcf4-d7ff9a7647be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:10.868370 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa0212-2f09-4e37-bcf4-d7ff9a7647be admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:10.868722 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa0212-2f09-4e37-bcf4-d7ff9a7647be admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:10.869067 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa0212-2f09-4e37-bcf4-d7ff9a7647be admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:10.869927 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fa0212-2f09-4e37-bcf4-d7ff9a7647be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:10.871730 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fa0212-2f09-4e37-bcf4-d7ff9a7647be admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:10.887181 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1330/2684] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:49:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:49:10.888403 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fea22fd-267e-4b55-8996-1bf1e4599553 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:10.898860 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fea22fd-267e-4b55-8996-1bf1e4599553 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:10.904356 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781edd95-b937-4fa6-b209-4ffdb8535699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:10.905035 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781edd95-b937-4fa6-b209-4ffdb8535699 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:10.905349 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781edd95-b937-4fa6-b209-4ffdb8535699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:10.905633 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781edd95-b937-4fa6-b209-4ffdb8535699 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:10.906429 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781edd95-b937-4fa6-b209-4ffdb8535699 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:10.908040 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781edd95-b937-4fa6-b209-4ffdb8535699 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:10.923799 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1355/2685] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:49:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:49:10.924949 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa0212-2f09-4e37-bcf4-d7ff9a7647be admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:10.935715 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa0212-2f09-4e37-bcf4-d7ff9a7647be admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:10.940408 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d71f99-95ae-4560-bc24-dd16c7bc1928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:10.941292 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d71f99-95ae-4560-bc24-dd16c7bc1928 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:10.941578 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d71f99-95ae-4560-bc24-dd16c7bc1928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:10.941890 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d71f99-95ae-4560-bc24-dd16c7bc1928 admin admin] PATH_INFO: `/v3/projects/643c56d46c1c4d21b4f5861b74e8737f/users/6e43987b825a482c892e268e58634e23/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:10.957537 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d71f99-95ae-4560-bc24-dd16c7bc1928 admin admin] RBAC: Authorizing `identity:create_grant(project_id=643c56d46c1c4d21b4f5861b74e8737f, user_id=6e43987b825a482c892e268e58634e23, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:10.960421 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d71f99-95ae-4560-bc24-dd16c7bc1928 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:10.975504 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1331/2686] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:49:10 2022] PUT /identity/v3/projects/643c56d46c1c4d21b4f5861b74e8737f/users/6e43987b825a482c892e268e58634e23/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:49:10.987011 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98233ca-1584-4326-8307-8bd3cc96f8ed None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:10.987300 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98233ca-1584-4326-8307-8bd3cc96f8ed None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:10.987586 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98233ca-1584-4326-8307-8bd3cc96f8ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:11.016088 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-a98233ca-1584-4326-8307-8bd3cc96f8ed None None] MFA Rules not processed for user `6e43987b825a482c892e268e58634e23`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:49:11.017330 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-a98233ca-1584-4326-8307-8bd3cc96f8ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:49:11.098884 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1356/2687] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:49:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:49:11.103845 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d71f99-95ae-4560-bc24-dd16c7bc1928 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:11.113602 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:49:11.183805 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d71f99-95ae-4560-bc24-dd16c7bc1928 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:11.188672 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc5fd37-918a-42b1-b6b2-f77b40e73eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:11.189539 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5fd37-918a-42b1-b6b2-f77b40e73eee admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:11.189860 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5fd37-918a-42b1-b6b2-f77b40e73eee admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:11.190138 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5fd37-918a-42b1-b6b2-f77b40e73eee admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:11.190841 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc5fd37-918a-42b1-b6b2-f77b40e73eee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:11.192743 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc5fd37-918a-42b1-b6b2-f77b40e73eee admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:11.224270 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1332/2688] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:49:11 2022] POST /identity/v3/projects => generated 375 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:49:11.225560 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98233ca-1584-4326-8307-8bd3cc96f8ed None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:11.238300 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98233ca-1584-4326-8307-8bd3cc96f8ed None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:11.242818 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:49:11.319476 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030bd56f-bfdf-4394-9268-e423e2828565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:11.320149 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030bd56f-bfdf-4394-9268-e423e2828565 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:11.320432 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030bd56f-bfdf-4394-9268-e423e2828565 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:11.320727 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030bd56f-bfdf-4394-9268-e423e2828565 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:11.321447 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030bd56f-bfdf-4394-9268-e423e2828565 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:11.323959 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030bd56f-bfdf-4394-9268-e423e2828565 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:11.347020 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1357/2689] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:49:11 2022] POST /identity/v3/users => generated 294 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:49:11.350351 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc5fd37-918a-42b1-b6b2-f77b40e73eee admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:11.362218 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc5fd37-918a-42b1-b6b2-f77b40e73eee admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:11.371911 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1659f46f-8ab8-4596-a219-50373fadf036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:11.371911 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659f46f-8ab8-4596-a219-50373fadf036 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:11.371911 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659f46f-8ab8-4596-a219-50373fadf036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:11.371911 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659f46f-8ab8-4596-a219-50373fadf036 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:11.372409 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1659f46f-8ab8-4596-a219-50373fadf036 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:11.372409 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1659f46f-8ab8-4596-a219-50373fadf036 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:11.384236 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1333/2690] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:49:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:49:11.388978 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030bd56f-bfdf-4394-9268-e423e2828565 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:11.400275 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030bd56f-bfdf-4394-9268-e423e2828565 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:11.405805 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ed3663-ee1d-46a3-9495-4922f8c7e418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:11.406708 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed3663-ee1d-46a3-9495-4922f8c7e418 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:11.406998 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed3663-ee1d-46a3-9495-4922f8c7e418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:11.407278 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed3663-ee1d-46a3-9495-4922f8c7e418 admin admin] PATH_INFO: `/v3/projects/aff934c1be004cc4b15505d44f9ec816/users/d1030fd5c48647beb90d8ad36cb5bafa/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:11.423958 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ed3663-ee1d-46a3-9495-4922f8c7e418 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aff934c1be004cc4b15505d44f9ec816, user_id=d1030fd5c48647beb90d8ad36cb5bafa, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:11.426732 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ed3663-ee1d-46a3-9495-4922f8c7e418 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:11.440346 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1358/2691] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:49:11 2022] PUT /identity/v3/projects/aff934c1be004cc4b15505d44f9ec816/users/d1030fd5c48647beb90d8ad36cb5bafa/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:49:11.444389 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1659f46f-8ab8-4596-a219-50373fadf036 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:11.455777 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1659f46f-8ab8-4596-a219-50373fadf036 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:11.512449 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1367808-4db2-4f83-8192-2e335d82980e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:11.513030 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1367808-4db2-4f83-8192-2e335d82980e admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:11.513317 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1367808-4db2-4f83-8192-2e335d82980e admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:11.513596 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1367808-4db2-4f83-8192-2e335d82980e admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:11.514354 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1367808-4db2-4f83-8192-2e335d82980e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:11.515943 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1367808-4db2-4f83-8192-2e335d82980e admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:11.525618 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1334/2692] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:49:11 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:49:11.530054 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ed3663-ee1d-46a3-9495-4922f8c7e418 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:11.540352 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ed3663-ee1d-46a3-9495-4922f8c7e418 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:11.545874 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3ca830-e35b-4c8f-84f0-e267470aaa2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:11.546749 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ca830-e35b-4c8f-84f0-e267470aaa2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:11.547035 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ca830-e35b-4c8f-84f0-e267470aaa2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:11.547313 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ca830-e35b-4c8f-84f0-e267470aaa2e admin admin] PATH_INFO: `/v3/projects/aff934c1be004cc4b15505d44f9ec816/users/d1030fd5c48647beb90d8ad36cb5bafa/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:11.549316 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3ca830-e35b-4c8f-84f0-e267470aaa2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=aff934c1be004cc4b15505d44f9ec816, user_id=d1030fd5c48647beb90d8ad36cb5bafa, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:11.552004 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3ca830-e35b-4c8f-84f0-e267470aaa2e admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:11.563065 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1359/2693] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:49:11 2022] PUT /identity/v3/projects/aff934c1be004cc4b15505d44f9ec816/users/d1030fd5c48647beb90d8ad36cb5bafa/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:49:11.574551 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d99e6-82ce-44fa-9ad9-e5ba24950ab5 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:11.574841 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d99e6-82ce-44fa-9ad9-e5ba24950ab5 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:11.575703 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d99e6-82ce-44fa-9ad9-e5ba24950ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:11.603690 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-4e0d99e6-82ce-44fa-9ad9-e5ba24950ab5 None None] MFA Rules not processed for user `d1030fd5c48647beb90d8ad36cb5bafa`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:49:11.605934 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-4e0d99e6-82ce-44fa-9ad9-e5ba24950ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:49:11.674701 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1335/2694] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:49:11 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:49:11.686235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3ca830-e35b-4c8f-84f0-e267470aaa2e admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:11.687969 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-fd3ca830-e35b-4c8f-84f0-e267470aaa2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:49:11.746489 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3ca830-e35b-4c8f-84f0-e267470aaa2e admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:11.750192 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0196e9c-dead-485c-801b-2b969e2b8e3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:11.750843 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0196e9c-dead-485c-801b-2b969e2b8e3c service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:11.751129 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0196e9c-dead-485c-801b-2b969e2b8e3c service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:11.751409 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0196e9c-dead-485c-801b-2b969e2b8e3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:11.812639 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0196e9c-dead-485c-801b-2b969e2b8e3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:11.813997 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0196e9c-dead-485c-801b-2b969e2b8e3c service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:11.823116 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1360/2695] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:49:11 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:49:13.832425 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0d99e6-82ce-44fa-9ad9-e5ba24950ab5 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:13.833901 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-4e0d99e6-82ce-44fa-9ad9-e5ba24950ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:49:13.839109 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:49:13.895368 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0d99e6-82ce-44fa-9ad9-e5ba24950ab5 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:13.898901 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b09dc2-0eb0-4be8-acda-555c6206279a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:13.899542 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b09dc2-0eb0-4be8-acda-555c6206279a service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:13.899824 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b09dc2-0eb0-4be8-acda-555c6206279a service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:13.900096 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b09dc2-0eb0-4be8-acda-555c6206279a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:13.902127 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-39b09dc2-0eb0-4be8-acda-555c6206279a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:49:13.965763 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b09dc2-0eb0-4be8-acda-555c6206279a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:13.967204 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b09dc2-0eb0-4be8-acda-555c6206279a service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:13.977049 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1336/2696] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:49:13 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:49:14.378913 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0196e9c-dead-485c-801b-2b969e2b8e3c service cinder] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:14.394181 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:49:14.451921 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0196e9c-dead-485c-801b-2b969e2b8e3c service cinder] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:14.457312 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067d3f8e-5004-4de1-bd02-55868f2b5c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:14.459095 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d3f8e-5004-4de1-bd02-55868f2b5c74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:14.459095 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d3f8e-5004-4de1-bd02-55868f2b5c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:14.459095 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d3f8e-5004-4de1-bd02-55868f2b5c74 admin admin] PATH_INFO: `/v3/users/6e43987b825a482c892e268e58634e23` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:14.460247 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067d3f8e-5004-4de1-bd02-55868f2b5c74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e43987b825a482c892e268e58634e23)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:14.462339 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067d3f8e-5004-4de1-bd02-55868f2b5c74 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:14.517092 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-067d3f8e-5004-4de1-bd02-55868f2b5c74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e43987b825a482c892e268e58634e23'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:14.530373 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-067d3f8e-5004-4de1-bd02-55868f2b5c74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e43987b825a482c892e268e58634e23'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:14.538959 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-067d3f8e-5004-4de1-bd02-55868f2b5c74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e43987b825a482c892e268e58634e23'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:14.539647 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-067d3f8e-5004-4de1-bd02-55868f2b5c74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e43987b825a482c892e268e58634e23'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:14.540252 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-067d3f8e-5004-4de1-bd02-55868f2b5c74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e43987b825a482c892e268e58634e23'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:14.553712 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1361/2697] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:49:14 2022] DELETE /identity/v3/users/6e43987b825a482c892e268e58634e23 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:49:14.558411 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd5aad-9a97-41f2-ac0e-bbb948460522 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:14.558950 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd5aad-9a97-41f2-ac0e-bbb948460522 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:14.559301 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd5aad-9a97-41f2-ac0e-bbb948460522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:14.578033 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-02dd5aad-9a97-41f2-ac0e-bbb948460522 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:49:14.579256 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-02dd5aad-9a97-41f2-ac0e-bbb948460522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:49:14.639797 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1337/2698] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:49:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:49:14.651198 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067d3f8e-5004-4de1-bd02-55868f2b5c74 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:14.653248 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-067d3f8e-5004-4de1-bd02-55868f2b5c74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:49:14.712784 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067d3f8e-5004-4de1-bd02-55868f2b5c74 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:14.716509 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e5411b-ae43-4bbb-a02d-1d7340983e12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:14.717158 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e5411b-ae43-4bbb-a02d-1d7340983e12 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:14.717440 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e5411b-ae43-4bbb-a02d-1d7340983e12 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:14.717740 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e5411b-ae43-4bbb-a02d-1d7340983e12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:14.727382 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e5411b-ae43-4bbb-a02d-1d7340983e12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:14.728807 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e5411b-ae43-4bbb-a02d-1d7340983e12 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:14.739906 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1362/2699] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:49:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:49:15.149471 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dd5aad-9a97-41f2-ac0e-bbb948460522 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:15.151062 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-02dd5aad-9a97-41f2-ac0e-bbb948460522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:49:15.165137 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dd5aad-9a97-41f2-ac0e-bbb948460522 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:15.172363 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874f20c7-a950-4c77-ad91-324e351c5857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:15.173248 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f20c7-a950-4c77-ad91-324e351c5857 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:15.173535 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f20c7-a950-4c77-ad91-324e351c5857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:15.173867 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f20c7-a950-4c77-ad91-324e351c5857 admin admin] PATH_INFO: `/v3/projects/643c56d46c1c4d21b4f5861b74e8737f` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:15.175205 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874f20c7-a950-4c77-ad91-324e351c5857 admin admin] RBAC: Authorizing `identity:delete_project(project_id=643c56d46c1c4d21b4f5861b74e8737f)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:15.177149 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874f20c7-a950-4c77-ad91-324e351c5857 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:15.231209 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:49:15.232497 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-874f20c7-a950-4c77-ad91-324e351c5857 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '643c56d46c1c4d21b4f5861b74e8737f', 'request_id': 'req-874f20c7-a950-4c77-ad91-324e351c5857'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:15.239825 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-874f20c7-a950-4c77-ad91-324e351c5857 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '643c56d46c1c4d21b4f5861b74e8737f', 'request_id': 'req-874f20c7-a950-4c77-ad91-324e351c5857'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:15.247478 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-874f20c7-a950-4c77-ad91-324e351c5857 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '643c56d46c1c4d21b4f5861b74e8737f', 'request_id': 'req-874f20c7-a950-4c77-ad91-324e351c5857'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:15.262635 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-874f20c7-a950-4c77-ad91-324e351c5857 admin admin] The token cache is being invalidate because project 643c56d46c1c4d21b4f5861b74e8737f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:49:15.263226 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-874f20c7-a950-4c77-ad91-324e351c5857 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:15.263883 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-874f20c7-a950-4c77-ad91-324e351c5857 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:15.265599 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1338/2700] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:49:15 2022] DELETE /identity/v3/projects/643c56d46c1c4d21b4f5861b74e8737f => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:49:15.269822 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e5411b-ae43-4bbb-a02d-1d7340983e12 service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:15.281101 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e5411b-ae43-4bbb-a02d-1d7340983e12 service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:15.316084 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:49:15.345931 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa0a7ba-c96c-47ea-8ae5-4564d4034c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:15.346817 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0a7ba-c96c-47ea-8ae5-4564d4034c64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:15.347098 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0a7ba-c96c-47ea-8ae5-4564d4034c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:15.347385 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0a7ba-c96c-47ea-8ae5-4564d4034c64 admin admin] PATH_INFO: `/v3/users/d1030fd5c48647beb90d8ad36cb5bafa` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:15.348717 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa0a7ba-c96c-47ea-8ae5-4564d4034c64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1030fd5c48647beb90d8ad36cb5bafa)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:15.350709 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa0a7ba-c96c-47ea-8ae5-4564d4034c64 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:15.415428 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-eaa0a7ba-c96c-47ea-8ae5-4564d4034c64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1030fd5c48647beb90d8ad36cb5bafa'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:15.432002 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-eaa0a7ba-c96c-47ea-8ae5-4564d4034c64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1030fd5c48647beb90d8ad36cb5bafa'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:15.442508 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-eaa0a7ba-c96c-47ea-8ae5-4564d4034c64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1030fd5c48647beb90d8ad36cb5bafa'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:15.443134 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-eaa0a7ba-c96c-47ea-8ae5-4564d4034c64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1030fd5c48647beb90d8ad36cb5bafa'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:15.443732 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-eaa0a7ba-c96c-47ea-8ae5-4564d4034c64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1030fd5c48647beb90d8ad36cb5bafa'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:15.457973 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1363/2701] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:49:15 2022] DELETE /identity/v3/users/d1030fd5c48647beb90d8ad36cb5bafa => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:49:15.852790 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874f20c7-a950-4c77-ad91-324e351c5857 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:15.855656 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-874f20c7-a950-4c77-ad91-324e351c5857 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:49:15.872154 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874f20c7-a950-4c77-ad91-324e351c5857 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:15.977710 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056497d9-cffe-40f1-8311-fe18d8259116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:15.978622 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056497d9-cffe-40f1-8311-fe18d8259116 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:15.978912 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056497d9-cffe-40f1-8311-fe18d8259116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:15.979193 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056497d9-cffe-40f1-8311-fe18d8259116 admin admin] PATH_INFO: `/v3/projects/aff934c1be004cc4b15505d44f9ec816` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:15.980702 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056497d9-cffe-40f1-8311-fe18d8259116 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aff934c1be004cc4b15505d44f9ec816)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:15.982718 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056497d9-cffe-40f1-8311-fe18d8259116 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:16.046331 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-056497d9-cffe-40f1-8311-fe18d8259116 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aff934c1be004cc4b15505d44f9ec816', 'request_id': 'req-056497d9-cffe-40f1-8311-fe18d8259116'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:16.054022 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-056497d9-cffe-40f1-8311-fe18d8259116 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aff934c1be004cc4b15505d44f9ec816', 'request_id': 'req-056497d9-cffe-40f1-8311-fe18d8259116'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:16.062204 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-056497d9-cffe-40f1-8311-fe18d8259116 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aff934c1be004cc4b15505d44f9ec816', 'request_id': 'req-056497d9-cffe-40f1-8311-fe18d8259116'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:16.079105 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-056497d9-cffe-40f1-8311-fe18d8259116 admin admin] The token cache is being invalidate because project aff934c1be004cc4b15505d44f9ec816 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:49:16.079473 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-056497d9-cffe-40f1-8311-fe18d8259116 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:16.080248 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-056497d9-cffe-40f1-8311-fe18d8259116 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:49:16.082110 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1339/2702] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:49:15 2022] DELETE /identity/v3/projects/aff934c1be004cc4b15505d44f9ec816 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:49:16.087263 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c7544-dd93-44ab-b30e-609711f84bbd None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:16.087556 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c7544-dd93-44ab-b30e-609711f84bbd None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:16.088598 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c7544-dd93-44ab-b30e-609711f84bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:16.113416 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-e61c7544-dd93-44ab-b30e-609711f84bbd None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:49:16.115464 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-e61c7544-dd93-44ab-b30e-609711f84bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:49:16.193882 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1364/2703] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:49:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:49:16.199683 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316226ef-6f38-4fc1-8251-4a65c9557b73 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:16.200280 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316226ef-6f38-4fc1-8251-4a65c9557b73 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:16.200663 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316226ef-6f38-4fc1-8251-4a65c9557b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:16.226956 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-316226ef-6f38-4fc1-8251-4a65c9557b73 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:49:16.228184 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-316226ef-6f38-4fc1-8251-4a65c9557b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:49:16.237566 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1340/2704] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:49:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:49:16.241802 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c7544-dd93-44ab-b30e-609711f84bbd None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:16.253012 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c7544-dd93-44ab-b30e-609711f84bbd None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:16.258379 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00cb440-5633-4d65-9f0b-79f9507437e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:16.259242 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00cb440-5633-4d65-9f0b-79f9507437e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:16.259541 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00cb440-5633-4d65-9f0b-79f9507437e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:16.259824 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00cb440-5633-4d65-9f0b-79f9507437e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:16.260771 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00cb440-5633-4d65-9f0b-79f9507437e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:49:16.261096 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00cb440-5633-4d65-9f0b-79f9507437e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:16.263404 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00cb440-5633-4d65-9f0b-79f9507437e3 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:16.281734 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1365/2705] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:49:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:49:16.283742 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316226ef-6f38-4fc1-8251-4a65c9557b73 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:16.295018 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316226ef-6f38-4fc1-8251-4a65c9557b73 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:16.299908 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a65c56f-71f5-4838-87ec-310356ed519c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:16.301442 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a65c56f-71f5-4838-87ec-310356ed519c admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:16.301442 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a65c56f-71f5-4838-87ec-310356ed519c admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:16.301442 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a65c56f-71f5-4838-87ec-310356ed519c admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:16.302215 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a65c56f-71f5-4838-87ec-310356ed519c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:16.304532 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a65c56f-71f5-4838-87ec-310356ed519c admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:16.339961 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1341/2706] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:49:16 2022] POST /identity/v3/projects => generated 379 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:49:16.341123 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00cb440-5633-4d65-9f0b-79f9507437e3 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:16.357374 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00cb440-5633-4d65-9f0b-79f9507437e3 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:16.405659 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:49:16.427675 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2815a020-e6ea-491f-92f7-3cefcde0695d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:16.429437 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2815a020-e6ea-491f-92f7-3cefcde0695d admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:16.430168 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2815a020-e6ea-491f-92f7-3cefcde0695d admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:16.430168 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2815a020-e6ea-491f-92f7-3cefcde0695d admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:16.430727 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2815a020-e6ea-491f-92f7-3cefcde0695d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:16.433208 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2815a020-e6ea-491f-92f7-3cefcde0695d admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:16.461976 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1366/2707] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:49:16 2022] POST /identity/v3/users => generated 290 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:49:16.463355 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a65c56f-71f5-4838-87ec-310356ed519c admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:16.481073 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a65c56f-71f5-4838-87ec-310356ed519c admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:16.487162 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c137b200-445a-4fd2-9a8c-633df3ffd002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:16.488008 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137b200-445a-4fd2-9a8c-633df3ffd002 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:16.488534 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137b200-445a-4fd2-9a8c-633df3ffd002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:16.488889 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137b200-445a-4fd2-9a8c-633df3ffd002 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:16.490440 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c137b200-445a-4fd2-9a8c-633df3ffd002 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:16.491932 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c137b200-445a-4fd2-9a8c-633df3ffd002 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:16.504972 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1342/2708] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:49:16 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:49:16.509105 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2815a020-e6ea-491f-92f7-3cefcde0695d admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:16.522666 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2815a020-e6ea-491f-92f7-3cefcde0695d admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:16.530285 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14e0417-fa0f-483a-8ae3-fbcacd6a7e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:16.531082 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e0417-fa0f-483a-8ae3-fbcacd6a7e35 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:16.531549 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e0417-fa0f-483a-8ae3-fbcacd6a7e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:16.531838 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e0417-fa0f-483a-8ae3-fbcacd6a7e35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:16.532589 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14e0417-fa0f-483a-8ae3-fbcacd6a7e35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:16.534259 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14e0417-fa0f-483a-8ae3-fbcacd6a7e35 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:16.546805 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1367/2709] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:49:16 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:49:16.551076 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c137b200-445a-4fd2-9a8c-633df3ffd002 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:16.564366 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c137b200-445a-4fd2-9a8c-633df3ffd002 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:16.570501 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621faf6f-20e9-44bb-873f-7511b921a1f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:16.571464 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621faf6f-20e9-44bb-873f-7511b921a1f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:16.571801 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621faf6f-20e9-44bb-873f-7511b921a1f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:16.572240 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621faf6f-20e9-44bb-873f-7511b921a1f8 admin admin] PATH_INFO: `/v3/projects/2fa7360005f4416f87703d3efb18d3e1/users/c5cd4e3fa19443a885d9dcb3e4bfcf5f/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:16.589822 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621faf6f-20e9-44bb-873f-7511b921a1f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fa7360005f4416f87703d3efb18d3e1, user_id=c5cd4e3fa19443a885d9dcb3e4bfcf5f, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:16.592586 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621faf6f-20e9-44bb-873f-7511b921a1f8 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:16.607069 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1343/2710] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:49:16 2022] PUT /identity/v3/projects/2fa7360005f4416f87703d3efb18d3e1/users/c5cd4e3fa19443a885d9dcb3e4bfcf5f/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:49:16.612305 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268c729-da3f-4fb9-bf7e-f0c3cd997830 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:16.612627 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268c729-da3f-4fb9-bf7e-f0c3cd997830 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:16.612939 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268c729-da3f-4fb9-bf7e-f0c3cd997830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:16.633141 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-a268c729-da3f-4fb9-bf7e-f0c3cd997830 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:49:16.634391 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-a268c729-da3f-4fb9-bf7e-f0c3cd997830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:49:16.700486 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1368/2711] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:49:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:49:16.710135 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621faf6f-20e9-44bb-873f-7511b921a1f8 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:16.711869 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-621faf6f-20e9-44bb-873f-7511b921a1f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:49:16.717763 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:49:16.775638 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621faf6f-20e9-44bb-873f-7511b921a1f8 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:16.779971 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be77ebe-ffb2-4815-8fc9-cb1020c7e816 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:16.780615 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be77ebe-ffb2-4815-8fc9-cb1020c7e816 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:16.780916 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be77ebe-ffb2-4815-8fc9-cb1020c7e816 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:16.781299 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be77ebe-ffb2-4815-8fc9-cb1020c7e816 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:16.791278 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be77ebe-ffb2-4815-8fc9-cb1020c7e816 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:16.792880 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be77ebe-ffb2-4815-8fc9-cb1020c7e816 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:16.803075 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1344/2712] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:49:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:49:20.948787 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0229480d-4b3c-4eb6-ad11-faa786c7abb6 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:20.949350 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0229480d-4b3c-4eb6-ad11-faa786c7abb6 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:20.949560 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0229480d-4b3c-4eb6-ad11-faa786c7abb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:20.978143 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-0229480d-4b3c-4eb6-ad11-faa786c7abb6 None None] MFA Rules not processed for user `c5cd4e3fa19443a885d9dcb3e4bfcf5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:49:20.979458 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-0229480d-4b3c-4eb6-ad11-faa786c7abb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:49:21.055778 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1369/2713] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:49:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:49:21.061005 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be77ebe-ffb2-4815-8fc9-cb1020c7e816 service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:21.072316 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be77ebe-ffb2-4815-8fc9-cb1020c7e816 service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:21.078333 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567dd2d4-ffb2-41ff-a46d-02323e3ab68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:21.079083 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567dd2d4-ffb2-41ff-a46d-02323e3ab68b admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:21.079373 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567dd2d4-ffb2-41ff-a46d-02323e3ab68b admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:21.079649 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567dd2d4-ffb2-41ff-a46d-02323e3ab68b admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:21.080354 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567dd2d4-ffb2-41ff-a46d-02323e3ab68b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:21.082311 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567dd2d4-ffb2-41ff-a46d-02323e3ab68b admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:21.110381 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1345/2714] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:49:21 2022] POST /identity/v3/projects => generated 381 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:49:21.115887 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0229480d-4b3c-4eb6-ad11-faa786c7abb6 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:21.129171 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0229480d-4b3c-4eb6-ad11-faa786c7abb6 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:21.184829 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:49:21.197735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d6baa8-26b4-449e-96b7-cbda6e91ed50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:21.198109 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d6baa8-26b4-449e-96b7-cbda6e91ed50 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:21.198452 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d6baa8-26b4-449e-96b7-cbda6e91ed50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:21.198784 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d6baa8-26b4-449e-96b7-cbda6e91ed50 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:21.199574 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d6baa8-26b4-449e-96b7-cbda6e91ed50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:21.201654 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d6baa8-26b4-449e-96b7-cbda6e91ed50 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:21.230440 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1370/2715] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:49:21 2022] POST /identity/v3/users => generated 297 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:49:21.231797 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567dd2d4-ffb2-41ff-a46d-02323e3ab68b admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:21.253432 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567dd2d4-ffb2-41ff-a46d-02323e3ab68b admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:21.261280 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999bddf2-234e-424d-8c98-54c632aaef9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:21.263033 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999bddf2-234e-424d-8c98-54c632aaef9f admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:21.263263 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999bddf2-234e-424d-8c98-54c632aaef9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:21.263777 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999bddf2-234e-424d-8c98-54c632aaef9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:21.265255 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999bddf2-234e-424d-8c98-54c632aaef9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:21.268263 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999bddf2-234e-424d-8c98-54c632aaef9f admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:21.295177 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1346/2716] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:49:21.300441 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d6baa8-26b4-449e-96b7-cbda6e91ed50 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:21.320751 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d6baa8-26b4-449e-96b7-cbda6e91ed50 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:21.331366 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03a994-a693-4b0d-a082-f625abb732bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:21.333006 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03a994-a693-4b0d-a082-f625abb732bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:21.333525 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03a994-a693-4b0d-a082-f625abb732bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:21.334088 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03a994-a693-4b0d-a082-f625abb732bb admin admin] PATH_INFO: `/v3/projects/f56229ac74a54bb093ea536523a01575/users/6ee0bcddb25e48b2a009419d6fdb3747/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:21.362717 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f03a994-a693-4b0d-a082-f625abb732bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f56229ac74a54bb093ea536523a01575, user_id=6ee0bcddb25e48b2a009419d6fdb3747, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:21.367674 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f03a994-a693-4b0d-a082-f625abb732bb admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:21.386482 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1371/2717] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:49:21 2022] PUT /identity/v3/projects/f56229ac74a54bb093ea536523a01575/users/6ee0bcddb25e48b2a009419d6fdb3747/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:49:21.390626 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999bddf2-234e-424d-8c98-54c632aaef9f admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:21.402292 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999bddf2-234e-424d-8c98-54c632aaef9f admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:21.467544 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b23a01-6f7c-48f0-a645-3c132ba5f6a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:21.468378 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b23a01-6f7c-48f0-a645-3c132ba5f6a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:21.468757 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b23a01-6f7c-48f0-a645-3c132ba5f6a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:21.469031 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b23a01-6f7c-48f0-a645-3c132ba5f6a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:21.469806 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b23a01-6f7c-48f0-a645-3c132ba5f6a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:21.471465 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b23a01-6f7c-48f0-a645-3c132ba5f6a8 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:21.482501 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1347/2718] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:49:21.486623 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03a994-a693-4b0d-a082-f625abb732bb admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:21.498628 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03a994-a693-4b0d-a082-f625abb732bb admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:21.503895 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c199a8a-1f90-4921-9742-624654fe1b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:21.504800 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c199a8a-1f90-4921-9742-624654fe1b8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:21.504800 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c199a8a-1f90-4921-9742-624654fe1b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:21.505392 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c199a8a-1f90-4921-9742-624654fe1b8b admin admin] PATH_INFO: `/v3/projects/f56229ac74a54bb093ea536523a01575/users/6ee0bcddb25e48b2a009419d6fdb3747/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:21.507437 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c199a8a-1f90-4921-9742-624654fe1b8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f56229ac74a54bb093ea536523a01575, user_id=6ee0bcddb25e48b2a009419d6fdb3747, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:21.510179 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c199a8a-1f90-4921-9742-624654fe1b8b admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:21.523747 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1372/2719] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:49:21 2022] PUT /identity/v3/projects/f56229ac74a54bb093ea536523a01575/users/6ee0bcddb25e48b2a009419d6fdb3747/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:49:26.295770 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381330d1-c74a-49b0-ab29-dc4483a2b957 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:26.296232 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381330d1-c74a-49b0-ab29-dc4483a2b957 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:26.296320 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381330d1-c74a-49b0-ab29-dc4483a2b957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:26.327367 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-381330d1-c74a-49b0-ab29-dc4483a2b957 None None] MFA Rules not processed for user `6ee0bcddb25e48b2a009419d6fdb3747`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:49:26.328906 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-381330d1-c74a-49b0-ab29-dc4483a2b957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:49:26.412102 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:49:26.425653 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1348/2720] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:49:26 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:49:26.438675 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c199a8a-1f90-4921-9742-624654fe1b8b admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:26.440602 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-3c199a8a-1f90-4921-9742-624654fe1b8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:49:26.510297 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c199a8a-1f90-4921-9742-624654fe1b8b admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:26.526729 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd043c7-39d0-42f3-a062-5798783a77f8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd15c5f6aeb874191b73c8ee0410c6bc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:26.528185 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd043c7-39d0-42f3-a062-5798783a77f8 service glance] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:26.528479 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd043c7-39d0-42f3-a062-5798783a77f8 service glance] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:26.532088 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd043c7-39d0-42f3-a062-5798783a77f8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:26.609316 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd043c7-39d0-42f3-a062-5798783a77f8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:26.610794 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd043c7-39d0-42f3-a062-5798783a77f8 service glance] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:26.622286 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1373/2721] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:49:26 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:49:26.735427 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb9208-ab14-4131-8837-c1097bbbf0e2 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:26.735795 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb9208-ab14-4131-8837-c1097bbbf0e2 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:26.736136 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb9208-ab14-4131-8837-c1097bbbf0e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:26.756777 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-acbb9208-ab14-4131-8837-c1097bbbf0e2 None None] MFA Rules not processed for user `85d9c4c57681447c898ebfd7990b05b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:49:26.758081 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-acbb9208-ab14-4131-8837-c1097bbbf0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:49:26.828694 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1349/2722] 10.222.0.13 () {58 vars in 1009 bytes} [Fri Jun 24 08:49:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:49:26.886156 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd043c7-39d0-42f3-a062-5798783a77f8 service glance] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:26.887742 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-cbd043c7-39d0-42f3-a062-5798783a77f8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:49:26.949974 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:49:26.966531 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd043c7-39d0-42f3-a062-5798783a77f8 service glance] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:26.971845 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d6fbd9-0ec0-42f4-a93c-d96f32cce836 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec8c2f113cf41a0939b81f38fbee813', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:26.972463 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6fbd9-0ec0-42f4-a93c-d96f32cce836 service swift] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:26.972752 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6fbd9-0ec0-42f4-a93c-d96f32cce836 service swift] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:26.973065 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6fbd9-0ec0-42f4-a93c-d96f32cce836 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:26.983012 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d6fbd9-0ec0-42f4-a93c-d96f32cce836 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:26.984901 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d6fbd9-0ec0-42f4-a93c-d96f32cce836 service swift] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:26.994176 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1374/2723] 10.222.0.13 () {60 vars in 1384 bytes} [Fri Jun 24 08:49:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 08:49:35.860178 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4204b342-e842-46ba-bed3-eff925f055cb None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:35.860558 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4204b342-e842-46ba-bed3-eff925f055cb None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:35.860744 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4204b342-e842-46ba-bed3-eff925f055cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:35.878189 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-4204b342-e842-46ba-bed3-eff925f055cb None None] MFA Rules not processed for user `c5cd4e3fa19443a885d9dcb3e4bfcf5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:49:35.879646 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-4204b342-e842-46ba-bed3-eff925f055cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:49:35.886540 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1350/2724] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:49:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:49:36.990510 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=6 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:49:38.269091 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f532c28-8563-4b46-be7f-0528e17010a6 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:38.269787 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f532c28-8563-4b46-be7f-0528e17010a6 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:38.269787 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f532c28-8563-4b46-be7f-0528e17010a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:38.291148 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-9f532c28-8563-4b46-be7f-0528e17010a6 None None] MFA Rules not processed for user `85d9c4c57681447c898ebfd7990b05b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:49:38.292459 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-9f532c28-8563-4b46-be7f-0528e17010a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:49:38.301171 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1375/2725] 10.222.0.13 () {58 vars in 1009 bytes} [Fri Jun 24 08:49:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:49:38.314271 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4204b342-e842-46ba-bed3-eff925f055cb None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:38.327712 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4204b342-e842-46ba-bed3-eff925f055cb None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:38.332957 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5b30ad-9c31-4b96-89c3-8143d6ac1a94 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec8c2f113cf41a0939b81f38fbee813', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:38.333608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b30ad-9c31-4b96-89c3-8143d6ac1a94 service swift] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:38.334105 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b30ad-9c31-4b96-89c3-8143d6ac1a94 service swift] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:38.334688 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b30ad-9c31-4b96-89c3-8143d6ac1a94 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:38.363535 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5b30ad-9c31-4b96-89c3-8143d6ac1a94 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:38.365162 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5b30ad-9c31-4b96-89c3-8143d6ac1a94 service swift] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:38.376353 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1351/2726] 10.222.0.13 () {60 vars in 1384 bytes} [Fri Jun 24 08:49:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 08:49:48.284048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:49:48.372365 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=37 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:49:55.370530 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8415c9a-f457-465d-9dec-e5ede0af53b7 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:55.371097 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8415c9a-f457-465d-9dec-e5ede0af53b7 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:55.371493 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8415c9a-f457-465d-9dec-e5ede0af53b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:55.393958 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-a8415c9a-f457-465d-9dec-e5ede0af53b7 None None] MFA Rules not processed for user `8d15a76041274e7381087bd21eb674c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:49:55.395541 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-a8415c9a-f457-465d-9dec-e5ede0af53b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:49:55.457974 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1376/2727] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:49:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:50:05.450721 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=25 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:50:06.896448 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5b30ad-9c31-4b96-89c3-8143d6ac1a94 service swift] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:50:06.958960 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5b30ad-9c31-4b96-89c3-8143d6ac1a94 service swift] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:50:06.964054 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e7fd8c-5744-4003-ba69-0fcbffc77bd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:50:06.964740 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7fd8c-5744-4003-ba69-0fcbffc77bd2 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:50:06.965058 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7fd8c-5744-4003-ba69-0fcbffc77bd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:50:06.965332 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7fd8c-5744-4003-ba69-0fcbffc77bd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:50:06.977063 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e7fd8c-5744-4003-ba69-0fcbffc77bd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:50:06.978563 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e7fd8c-5744-4003-ba69-0fcbffc77bd2 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:50:06.990617 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1352/2728] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:50:06 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:50:16.985762 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=26 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:50:33.752163 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8415c9a-f457-465d-9dec-e5ede0af53b7 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:50:33.764167 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8415c9a-f457-465d-9dec-e5ede0af53b7 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:50:33.768701 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ece719-1a52-47f0-a9eb-fda20596434c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:50:33.769371 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ece719-1a52-47f0-a9eb-fda20596434c service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:50:33.769655 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ece719-1a52-47f0-a9eb-fda20596434c service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:50:33.769958 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ece719-1a52-47f0-a9eb-fda20596434c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:50:33.771866 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-72ece719-1a52-47f0-a9eb-fda20596434c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:50:33.851211 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ece719-1a52-47f0-a9eb-fda20596434c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:50:33.852675 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ece719-1a52-47f0-a9eb-fda20596434c service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:50:33.865513 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1377/2729] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:50:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:50:35.161010 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e7fd8c-5744-4003-ba69-0fcbffc77bd2 service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:50:35.170278 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e7fd8c-5744-4003-ba69-0fcbffc77bd2 service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:50:35.173992 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab5af26-77bf-429c-a6a4-abdaa4df95d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:50:35.174635 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab5af26-77bf-429c-a6a4-abdaa4df95d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:50:35.174918 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab5af26-77bf-429c-a6a4-abdaa4df95d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:50:35.175195 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab5af26-77bf-429c-a6a4-abdaa4df95d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:50:35.177041 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-4ab5af26-77bf-429c-a6a4-abdaa4df95d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:50:35.192639 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab5af26-77bf-429c-a6a4-abdaa4df95d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:50:35.194418 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab5af26-77bf-429c-a6a4-abdaa4df95d5 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:50:35.209393 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1353/2730] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:50:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:50:43.860186 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=34 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:50:45.201401 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=6 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:51:01.430176 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da5d56-52ef-4b77-8cdd-66567eed6823 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:01.430931 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da5d56-52ef-4b77-8cdd-66567eed6823 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:01.431036 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da5d56-52ef-4b77-8cdd-66567eed6823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:01.455106 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-e6da5d56-52ef-4b77-8cdd-66567eed6823 None None] MFA Rules not processed for user `8d15a76041274e7381087bd21eb674c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:51:01.456683 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-e6da5d56-52ef-4b77-8cdd-66567eed6823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:51:01.468516 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1378/2731] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:51:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:51:06.520035 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab5af26-77bf-429c-a6a4-abdaa4df95d5 service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:51:06.537750 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab5af26-77bf-429c-a6a4-abdaa4df95d5 service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:51:06.544608 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3355f5f5-d084-4d4a-8519-4e9e4a45874e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:51:06.548855 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355f5f5-d084-4d4a-8519-4e9e4a45874e service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:06.548855 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355f5f5-d084-4d4a-8519-4e9e4a45874e service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:06.548855 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355f5f5-d084-4d4a-8519-4e9e4a45874e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:06.551539 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-3355f5f5-d084-4d4a-8519-4e9e4a45874e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:51:06.572827 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3355f5f5-d084-4d4a-8519-4e9e4a45874e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:51:06.574237 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3355f5f5-d084-4d4a-8519-4e9e4a45874e service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:51:06.593802 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1354/2732] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:51:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:51:09.818572 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6da5d56-52ef-4b77-8cdd-66567eed6823 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:51:09.820183 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-e6da5d56-52ef-4b77-8cdd-66567eed6823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:51:09.833889 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6da5d56-52ef-4b77-8cdd-66567eed6823 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:51:09.839084 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd4bd57-f368-4d34-9a40-d1117d077d38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:51:09.839945 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd4bd57-f368-4d34-9a40-d1117d077d38 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:09.840233 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd4bd57-f368-4d34-9a40-d1117d077d38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:09.840505 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd4bd57-f368-4d34-9a40-d1117d077d38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:09.842388 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-6cd4bd57-f368-4d34-9a40-d1117d077d38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:51:09.925047 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd4bd57-f368-4d34-9a40-d1117d077d38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:51:09.926589 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd4bd57-f368-4d34-9a40-d1117d077d38 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:51:09.938684 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1379/2733] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:51:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:51:16.582244 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=6 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:51:17.255106 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3355f5f5-d084-4d4a-8519-4e9e4a45874e service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:51:17.257278 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-3355f5f5-d084-4d4a-8519-4e9e4a45874e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:51:17.276658 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3355f5f5-d084-4d4a-8519-4e9e4a45874e service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:51:17.281710 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a163eefa-3d14-4563-a4f5-db224bf430bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:51:17.282613 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163eefa-3d14-4563-a4f5-db224bf430bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:17.282941 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163eefa-3d14-4563-a4f5-db224bf430bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:17.283218 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163eefa-3d14-4563-a4f5-db224bf430bc admin admin] PATH_INFO: `/v3/users/bf905723382e4372ae63d77a376982ff` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:17.284434 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a163eefa-3d14-4563-a4f5-db224bf430bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf905723382e4372ae63d77a376982ff)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:51:17.286425 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a163eefa-3d14-4563-a4f5-db224bf430bc admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:51:17.336918 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-a163eefa-3d14-4563-a4f5-db224bf430bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf905723382e4372ae63d77a376982ff'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:51:17.349828 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-a163eefa-3d14-4563-a4f5-db224bf430bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf905723382e4372ae63d77a376982ff'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:51:17.350521 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-a163eefa-3d14-4563-a4f5-db224bf430bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf905723382e4372ae63d77a376982ff'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:51:17.361515 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-a163eefa-3d14-4563-a4f5-db224bf430bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf905723382e4372ae63d77a376982ff'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:51:17.362266 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-a163eefa-3d14-4563-a4f5-db224bf430bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf905723382e4372ae63d77a376982ff'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:51:17.377871 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1355/2734] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:51:17 2022] DELETE /identity/v3/users/bf905723382e4372ae63d77a376982ff => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:51:17.560150 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd4bd57-f368-4d34-9a40-d1117d077d38 service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:51:17.570721 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd4bd57-f368-4d34-9a40-d1117d077d38 service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:51:17.640250 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4974ab-8b86-4903-baf2-905df3ca15ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:51:17.641112 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4974ab-8b86-4903-baf2-905df3ca15ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:17.641399 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4974ab-8b86-4903-baf2-905df3ca15ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:17.641678 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4974ab-8b86-4903-baf2-905df3ca15ed admin admin] PATH_INFO: `/v3/projects/d0b95979aebe47bc92c387de59394942` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:17.642900 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4974ab-8b86-4903-baf2-905df3ca15ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0b95979aebe47bc92c387de59394942)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:51:17.644820 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4974ab-8b86-4903-baf2-905df3ca15ed admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:51:17.692544 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-bc4974ab-8b86-4903-baf2-905df3ca15ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0b95979aebe47bc92c387de59394942', 'request_id': 'req-bc4974ab-8b86-4903-baf2-905df3ca15ed'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:51:17.699369 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-bc4974ab-8b86-4903-baf2-905df3ca15ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0b95979aebe47bc92c387de59394942', 'request_id': 'req-bc4974ab-8b86-4903-baf2-905df3ca15ed'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:51:17.704906 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-bc4974ab-8b86-4903-baf2-905df3ca15ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0b95979aebe47bc92c387de59394942', 'request_id': 'req-bc4974ab-8b86-4903-baf2-905df3ca15ed'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:51:17.718298 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-bc4974ab-8b86-4903-baf2-905df3ca15ed admin admin] The token cache is being invalidate because project d0b95979aebe47bc92c387de59394942 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:51:17.718591 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-bc4974ab-8b86-4903-baf2-905df3ca15ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:51:17.719236 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-bc4974ab-8b86-4903-baf2-905df3ca15ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:51:17.720791 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1380/2735] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:51:17 2022] DELETE /identity/v3/projects/d0b95979aebe47bc92c387de59394942 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:51:17.724830 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a163eefa-3d14-4563-a4f5-db224bf430bc admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:51:17.726287 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-a163eefa-3d14-4563-a4f5-db224bf430bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:51:17.739799 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a163eefa-3d14-4563-a4f5-db224bf430bc admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:51:17.799011 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a83190f-980d-4697-8d5d-31ce8969fe7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:51:17.799889 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83190f-980d-4697-8d5d-31ce8969fe7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:17.800174 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83190f-980d-4697-8d5d-31ce8969fe7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:17.800450 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83190f-980d-4697-8d5d-31ce8969fe7d admin admin] PATH_INFO: `/v3/users/ce24dd074ae44e4ea79ccf6451634626` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:17.801725 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a83190f-980d-4697-8d5d-31ce8969fe7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce24dd074ae44e4ea79ccf6451634626)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:51:17.803636 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a83190f-980d-4697-8d5d-31ce8969fe7d admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:51:17.852436 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-0a83190f-980d-4697-8d5d-31ce8969fe7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce24dd074ae44e4ea79ccf6451634626'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:51:17.866525 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-0a83190f-980d-4697-8d5d-31ce8969fe7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce24dd074ae44e4ea79ccf6451634626'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:51:17.867238 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-0a83190f-980d-4697-8d5d-31ce8969fe7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce24dd074ae44e4ea79ccf6451634626'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:51:17.875458 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-0a83190f-980d-4697-8d5d-31ce8969fe7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce24dd074ae44e4ea79ccf6451634626'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:51:17.876128 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-0a83190f-980d-4697-8d5d-31ce8969fe7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce24dd074ae44e4ea79ccf6451634626'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:51:17.877831 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:51:17.893203 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1356/2736] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:51:17 2022] DELETE /identity/v3/users/ce24dd074ae44e4ea79ccf6451634626 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:51:18.082108 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4974ab-8b86-4903-baf2-905df3ca15ed admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:51:18.083835 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-bc4974ab-8b86-4903-baf2-905df3ca15ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:51:18.102697 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4974ab-8b86-4903-baf2-905df3ca15ed admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:51:18.121010 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:51:18.171509 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2806ae8-947d-4ac9-8bbc-61863b49e707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:51:18.172376 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2806ae8-947d-4ac9-8bbc-61863b49e707 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:18.172672 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2806ae8-947d-4ac9-8bbc-61863b49e707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:18.172949 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2806ae8-947d-4ac9-8bbc-61863b49e707 admin admin] PATH_INFO: `/v3/projects/81aa67602e4545b0983e9e201551e578` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:18.174178 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2806ae8-947d-4ac9-8bbc-61863b49e707 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81aa67602e4545b0983e9e201551e578)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:51:18.176109 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2806ae8-947d-4ac9-8bbc-61863b49e707 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:51:18.225096 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c2806ae8-947d-4ac9-8bbc-61863b49e707 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81aa67602e4545b0983e9e201551e578', 'request_id': 'req-c2806ae8-947d-4ac9-8bbc-61863b49e707'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:51:18.233549 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c2806ae8-947d-4ac9-8bbc-61863b49e707 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81aa67602e4545b0983e9e201551e578', 'request_id': 'req-c2806ae8-947d-4ac9-8bbc-61863b49e707'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:51:18.240171 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c2806ae8-947d-4ac9-8bbc-61863b49e707 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81aa67602e4545b0983e9e201551e578', 'request_id': 'req-c2806ae8-947d-4ac9-8bbc-61863b49e707'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:51:18.255145 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c2806ae8-947d-4ac9-8bbc-61863b49e707 admin admin] The token cache is being invalidate because project 81aa67602e4545b0983e9e201551e578 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:51:18.255433 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c2806ae8-947d-4ac9-8bbc-61863b49e707 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:51:18.256317 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c2806ae8-947d-4ac9-8bbc-61863b49e707 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:51:18.257951 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1381/2737] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:51:18 2022] DELETE /identity/v3/projects/81aa67602e4545b0983e9e201551e578 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:51:27.679189 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6bd6e9-4926-40b8-9be5-11acc59dca4f None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:27.680767 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6bd6e9-4926-40b8-9be5-11acc59dca4f None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:27.680767 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6bd6e9-4926-40b8-9be5-11acc59dca4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:27.698071 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-dc6bd6e9-4926-40b8-9be5-11acc59dca4f None None] MFA Rules not processed for user `85d9c4c57681447c898ebfd7990b05b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:51:27.699280 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-dc6bd6e9-4926-40b8-9be5-11acc59dca4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:51:27.756069 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1357/2738] 10.222.0.13 () {58 vars in 967 bytes} [Fri Jun 24 08:51:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:51:27.796770 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2806ae8-947d-4ac9-8bbc-61863b49e707 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:51:27.798266 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-c2806ae8-947d-4ac9-8bbc-61863b49e707 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:51:27.867990 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2806ae8-947d-4ac9-8bbc-61863b49e707 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:51:27.873106 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ed4a9-a131-4745-87f2-1705286a6ce5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bec8c2f113cf41a0939b81f38fbee813', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:51:27.873772 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ed4a9-a131-4745-87f2-1705286a6ce5 service swift] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:27.874057 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ed4a9-a131-4745-87f2-1705286a6ce5 service swift] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:27.874333 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ed4a9-a131-4745-87f2-1705286a6ce5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:27.883343 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957ed4a9-a131-4745-87f2-1705286a6ce5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:51:27.884884 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957ed4a9-a131-4745-87f2-1705286a6ce5 service swift] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:51:27.893161 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1382/2739] 10.222.0.13 () {60 vars in 1384 bytes} [Fri Jun 24 08:51:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 08:51:31.304544 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6bd6e9-4926-40b8-9be5-11acc59dca4f None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:51:31.306268 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-dc6bd6e9-4926-40b8-9be5-11acc59dca4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:51:31.416936 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6bd6e9-4926-40b8-9be5-11acc59dca4f None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:51:31.422537 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90063a9d-b542-488e-ab7e-aa1588993b96 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39cc268bad604dd08594e3b39c7ecf5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:51:31.423199 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90063a9d-b542-488e-ab7e-aa1588993b96 service placement] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:31.423480 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90063a9d-b542-488e-ab7e-aa1588993b96 service placement] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:31.423754 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90063a9d-b542-488e-ab7e-aa1588993b96 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:31.425612 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-90063a9d-b542-488e-ab7e-aa1588993b96 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:51:31.443157 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90063a9d-b542-488e-ab7e-aa1588993b96 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:51:31.444794 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90063a9d-b542-488e-ab7e-aa1588993b96 service placement] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:51:31.457626 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1358/2740] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:51:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:51:34.926985 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ed4a9-a131-4745-87f2-1705286a6ce5 service swift] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:51:34.928602 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-957ed4a9-a131-4745-87f2-1705286a6ce5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:51:34.983990 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:51:35.018682 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ed4a9-a131-4745-87f2-1705286a6ce5 service swift] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:51:35.024380 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b3bccc-7ea9-4267-9a17-d8ad5ce235c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:51:35.025386 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3bccc-7ea9-4267-9a17-d8ad5ce235c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:35.025801 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3bccc-7ea9-4267-9a17-d8ad5ce235c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:35.026105 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3bccc-7ea9-4267-9a17-d8ad5ce235c7 admin admin] PATH_INFO: `/v3/users/c5cd4e3fa19443a885d9dcb3e4bfcf5f` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:35.027401 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b3bccc-7ea9-4267-9a17-d8ad5ce235c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5cd4e3fa19443a885d9dcb3e4bfcf5f)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:51:35.029506 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b3bccc-7ea9-4267-9a17-d8ad5ce235c7 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:51:35.090967 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c2b3bccc-7ea9-4267-9a17-d8ad5ce235c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5cd4e3fa19443a885d9dcb3e4bfcf5f'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:51:35.107214 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c2b3bccc-7ea9-4267-9a17-d8ad5ce235c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5cd4e3fa19443a885d9dcb3e4bfcf5f'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:51:35.115973 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c2b3bccc-7ea9-4267-9a17-d8ad5ce235c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5cd4e3fa19443a885d9dcb3e4bfcf5f'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:51:35.117374 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c2b3bccc-7ea9-4267-9a17-d8ad5ce235c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5cd4e3fa19443a885d9dcb3e4bfcf5f'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:51:35.118490 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c2b3bccc-7ea9-4267-9a17-d8ad5ce235c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5cd4e3fa19443a885d9dcb3e4bfcf5f'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:51:35.135686 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1383/2741] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:51:34 2022] DELETE /identity/v3/users/c5cd4e3fa19443a885d9dcb3e4bfcf5f => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:51:35.322172 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90063a9d-b542-488e-ab7e-aa1588993b96 service placement] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:51:35.335098 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90063a9d-b542-488e-ab7e-aa1588993b96 service placement] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:51:35.415976 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393e9739-eb59-4238-bfe7-0c0b174882c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:51:35.416856 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393e9739-eb59-4238-bfe7-0c0b174882c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:35.417145 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393e9739-eb59-4238-bfe7-0c0b174882c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:35.417423 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393e9739-eb59-4238-bfe7-0c0b174882c6 admin admin] PATH_INFO: `/v3/projects/2fa7360005f4416f87703d3efb18d3e1` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:35.418784 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393e9739-eb59-4238-bfe7-0c0b174882c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fa7360005f4416f87703d3efb18d3e1)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:51:35.420718 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393e9739-eb59-4238-bfe7-0c0b174882c6 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:51:35.456163 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:51:35.476830 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-393e9739-eb59-4238-bfe7-0c0b174882c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fa7360005f4416f87703d3efb18d3e1', 'request_id': 'req-393e9739-eb59-4238-bfe7-0c0b174882c6'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:51:35.483405 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-393e9739-eb59-4238-bfe7-0c0b174882c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fa7360005f4416f87703d3efb18d3e1', 'request_id': 'req-393e9739-eb59-4238-bfe7-0c0b174882c6'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:51:35.488979 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-393e9739-eb59-4238-bfe7-0c0b174882c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fa7360005f4416f87703d3efb18d3e1', 'request_id': 'req-393e9739-eb59-4238-bfe7-0c0b174882c6'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:51:35.504840 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-393e9739-eb59-4238-bfe7-0c0b174882c6 admin admin] The token cache is being invalidate because project 2fa7360005f4416f87703d3efb18d3e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:51:35.505488 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-393e9739-eb59-4238-bfe7-0c0b174882c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:51:35.506224 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-393e9739-eb59-4238-bfe7-0c0b174882c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:51:35.511800 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1359/2742] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:51:35 2022] DELETE /identity/v3/projects/2fa7360005f4416f87703d3efb18d3e1 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:51:35.513474 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b3bccc-7ea9-4267-9a17-d8ad5ce235c7 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:51:35.515132 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-c2b3bccc-7ea9-4267-9a17-d8ad5ce235c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:51:35.531056 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b3bccc-7ea9-4267-9a17-d8ad5ce235c7 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:51:35.596204 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240ff823-957b-44c7-a010-8c514a9a1597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:51:35.597162 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240ff823-957b-44c7-a010-8c514a9a1597 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:35.597445 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240ff823-957b-44c7-a010-8c514a9a1597 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:35.597750 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240ff823-957b-44c7-a010-8c514a9a1597 admin admin] PATH_INFO: `/v3/users/6ee0bcddb25e48b2a009419d6fdb3747` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:35.598972 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240ff823-957b-44c7-a010-8c514a9a1597 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ee0bcddb25e48b2a009419d6fdb3747)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:51:35.601544 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240ff823-957b-44c7-a010-8c514a9a1597 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:51:35.662520 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-240ff823-957b-44c7-a010-8c514a9a1597 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ee0bcddb25e48b2a009419d6fdb3747'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:51:35.672673 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:51:35.678733 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-240ff823-957b-44c7-a010-8c514a9a1597 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ee0bcddb25e48b2a009419d6fdb3747'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:51:35.688711 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-240ff823-957b-44c7-a010-8c514a9a1597 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ee0bcddb25e48b2a009419d6fdb3747'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:51:35.689746 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-240ff823-957b-44c7-a010-8c514a9a1597 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ee0bcddb25e48b2a009419d6fdb3747'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:51:35.690535 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-240ff823-957b-44c7-a010-8c514a9a1597 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ee0bcddb25e48b2a009419d6fdb3747'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:51:35.704270 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1384/2743] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:51:35 2022] DELETE /identity/v3/users/6ee0bcddb25e48b2a009419d6fdb3747 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:51:35.910216 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393e9739-eb59-4238-bfe7-0c0b174882c6 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:51:35.911861 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-393e9739-eb59-4238-bfe7-0c0b174882c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:51:35.926131 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393e9739-eb59-4238-bfe7-0c0b174882c6 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:51:36.002702 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c86a53-af4d-45ed-8760-845daee08842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:51:36.004227 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c86a53-af4d-45ed-8760-845daee08842 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:36.004678 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c86a53-af4d-45ed-8760-845daee08842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:36.004969 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c86a53-af4d-45ed-8760-845daee08842 admin admin] PATH_INFO: `/v3/projects/f56229ac74a54bb093ea536523a01575` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:36.006856 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c86a53-af4d-45ed-8760-845daee08842 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f56229ac74a54bb093ea536523a01575)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:51:36.009949 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c86a53-af4d-45ed-8760-845daee08842 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:51:36.070792 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-65c86a53-af4d-45ed-8760-845daee08842 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f56229ac74a54bb093ea536523a01575', 'request_id': 'req-65c86a53-af4d-45ed-8760-845daee08842'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:51:36.079663 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-65c86a53-af4d-45ed-8760-845daee08842 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f56229ac74a54bb093ea536523a01575', 'request_id': 'req-65c86a53-af4d-45ed-8760-845daee08842'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:51:36.088181 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-65c86a53-af4d-45ed-8760-845daee08842 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f56229ac74a54bb093ea536523a01575', 'request_id': 'req-65c86a53-af4d-45ed-8760-845daee08842'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:51:36.101720 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-65c86a53-af4d-45ed-8760-845daee08842 admin admin] The token cache is being invalidate because project f56229ac74a54bb093ea536523a01575 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:51:36.102044 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-65c86a53-af4d-45ed-8760-845daee08842 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:51:36.102766 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-65c86a53-af4d-45ed-8760-845daee08842 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:51:36.104696 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1360/2744] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:51:35 2022] DELETE /identity/v3/projects/f56229ac74a54bb093ea536523a01575 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:51:36.110828 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409ed91-b785-4114-815f-dfb0aa4fc5fc None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:36.111362 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409ed91-b785-4114-815f-dfb0aa4fc5fc None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:36.111460 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409ed91-b785-4114-815f-dfb0aa4fc5fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:36.131657 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-3409ed91-b785-4114-815f-dfb0aa4fc5fc None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:51:36.133365 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-3409ed91-b785-4114-815f-dfb0aa4fc5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:51:36.212309 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1385/2745] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:51:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:51:36.220490 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe3ce74-d744-4e49-a294-7de38b767573 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:36.220823 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe3ce74-d744-4e49-a294-7de38b767573 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:36.221078 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe3ce74-d744-4e49-a294-7de38b767573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:36.242159 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-ffe3ce74-d744-4e49-a294-7de38b767573 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:51:36.243583 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-ffe3ce74-d744-4e49-a294-7de38b767573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:51:36.252462 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1361/2746] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:51:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:51:36.257980 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3409ed91-b785-4114-815f-dfb0aa4fc5fc None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:51:36.269089 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3409ed91-b785-4114-815f-dfb0aa4fc5fc None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:51:36.274423 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16ae346-c2a8-45cf-a597-4a8455786e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:51:36.275132 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ae346-c2a8-45cf-a597-4a8455786e67 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:36.275532 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ae346-c2a8-45cf-a597-4a8455786e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:36.275693 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ae346-c2a8-45cf-a597-4a8455786e67 admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:36.276522 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16ae346-c2a8-45cf-a597-4a8455786e67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:51:36.276939 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16ae346-c2a8-45cf-a597-4a8455786e67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:51:36.278773 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16ae346-c2a8-45cf-a597-4a8455786e67 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:51:36.292426 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1386/2747] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:51:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:51:36.299995 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe3ce74-d744-4e49-a294-7de38b767573 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:51:36.311065 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe3ce74-d744-4e49-a294-7de38b767573 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:51:36.316212 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090bb1a7-5435-4606-91b3-85e5ef4e48a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:51:36.316880 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090bb1a7-5435-4606-91b3-85e5ef4e48a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:36.317157 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090bb1a7-5435-4606-91b3-85e5ef4e48a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:36.317427 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090bb1a7-5435-4606-91b3-85e5ef4e48a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:36.318166 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090bb1a7-5435-4606-91b3-85e5ef4e48a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:51:36.321013 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090bb1a7-5435-4606-91b3-85e5ef4e48a1 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:51:36.352172 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1362/2748] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:51:36 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:51:36.356587 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16ae346-c2a8-45cf-a597-4a8455786e67 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:51:36.370348 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16ae346-c2a8-45cf-a597-4a8455786e67 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:51:36.434237 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3410b44-1ee6-423c-ad23-edcd3e62a212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:51:36.434941 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3410b44-1ee6-423c-ad23-edcd3e62a212 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:36.435223 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3410b44-1ee6-423c-ad23-edcd3e62a212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:36.435498 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3410b44-1ee6-423c-ad23-edcd3e62a212 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:36.436197 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3410b44-1ee6-423c-ad23-edcd3e62a212 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:51:36.438134 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3410b44-1ee6-423c-ad23-edcd3e62a212 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:51:36.460446 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1387/2749] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:51:36 2022] POST /identity/v3/users => generated 286 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:51:36.464275 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090bb1a7-5435-4606-91b3-85e5ef4e48a1 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:51:36.476839 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090bb1a7-5435-4606-91b3-85e5ef4e48a1 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:51:36.486606 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df701e-4b01-4a94-8667-5fede19026de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:51:36.487369 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df701e-4b01-4a94-8667-5fede19026de admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:36.487695 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df701e-4b01-4a94-8667-5fede19026de admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:36.488015 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df701e-4b01-4a94-8667-5fede19026de admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:36.488833 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2df701e-4b01-4a94-8667-5fede19026de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:51:36.490552 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2df701e-4b01-4a94-8667-5fede19026de admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:51:36.503170 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1363/2750] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:51:36 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:51:36.508655 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3410b44-1ee6-423c-ad23-edcd3e62a212 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:51:36.519657 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3410b44-1ee6-423c-ad23-edcd3e62a212 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:51:36.525924 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d553304e-321d-452c-a437-4f2ea7963e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:51:36.526697 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d553304e-321d-452c-a437-4f2ea7963e25 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:36.527032 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d553304e-321d-452c-a437-4f2ea7963e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:36.527314 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d553304e-321d-452c-a437-4f2ea7963e25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:36.528093 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d553304e-321d-452c-a437-4f2ea7963e25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:51:36.529993 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d553304e-321d-452c-a437-4f2ea7963e25 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:51:36.542423 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1388/2751] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:51:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:51:36.547848 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df701e-4b01-4a94-8667-5fede19026de admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:51:36.562208 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df701e-4b01-4a94-8667-5fede19026de admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:51:36.568232 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a585092a-1e68-420c-b0ef-c4de6268e885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:51:36.569155 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a585092a-1e68-420c-b0ef-c4de6268e885 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:36.569439 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a585092a-1e68-420c-b0ef-c4de6268e885 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:36.569857 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a585092a-1e68-420c-b0ef-c4de6268e885 admin admin] PATH_INFO: `/v3/projects/33c8998057e6449e9c881b1b279faadd/users/038c41319feb49a486832af853c332b1/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:36.573122 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:51:36.593249 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a585092a-1e68-420c-b0ef-c4de6268e885 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33c8998057e6449e9c881b1b279faadd, user_id=038c41319feb49a486832af853c332b1, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:51:36.599207 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a585092a-1e68-420c-b0ef-c4de6268e885 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:51:36.622692 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1364/2752] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:51:36 2022] PUT /identity/v3/projects/33c8998057e6449e9c881b1b279faadd/users/038c41319feb49a486832af853c332b1/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:51:36.629050 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc98420-9c44-40dc-9850-6903652833a9 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:36.629438 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc98420-9c44-40dc-9850-6903652833a9 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:36.629872 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc98420-9c44-40dc-9850-6903652833a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:36.652513 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-0cc98420-9c44-40dc-9850-6903652833a9 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:51:36.653959 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-0cc98420-9c44-40dc-9850-6903652833a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:51:36.708526 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:51:36.742668 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1389/2753] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:51:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:51:36.753194 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a585092a-1e68-420c-b0ef-c4de6268e885 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:51:36.755718 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-a585092a-1e68-420c-b0ef-c4de6268e885 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:51:36.836240 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a585092a-1e68-420c-b0ef-c4de6268e885 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:51:36.840249 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b88242-5087-47a3-bcb4-4b2cf3dadd7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:51:36.840902 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b88242-5087-47a3-bcb4-4b2cf3dadd7e service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:36.841193 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b88242-5087-47a3-bcb4-4b2cf3dadd7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:36.841466 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b88242-5087-47a3-bcb4-4b2cf3dadd7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:36.852157 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b88242-5087-47a3-bcb4-4b2cf3dadd7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:51:36.853836 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b88242-5087-47a3-bcb4-4b2cf3dadd7e service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:51:36.864609 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1365/2754] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:51:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:51:41.736698 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353131f7-8eb7-46fd-a9ee-29feb49890ac None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:41.737841 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353131f7-8eb7-46fd-a9ee-29feb49890ac None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:41.737841 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353131f7-8eb7-46fd-a9ee-29feb49890ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:41.768097 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-353131f7-8eb7-46fd-a9ee-29feb49890ac None None] MFA Rules not processed for user `038c41319feb49a486832af853c332b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:51:41.769354 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-353131f7-8eb7-46fd-a9ee-29feb49890ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:51:41.865913 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1390/2755] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:51:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:51:41.872312 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b88242-5087-47a3-bcb4-4b2cf3dadd7e service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:51:41.884154 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b88242-5087-47a3-bcb4-4b2cf3dadd7e service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:51:41.890465 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3184a9a-9986-41d9-88a7-44b57bcb6ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:51:41.891448 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3184a9a-9986-41d9-88a7-44b57bcb6ae2 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:41.891820 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3184a9a-9986-41d9-88a7-44b57bcb6ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:41.892106 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3184a9a-9986-41d9-88a7-44b57bcb6ae2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:41.893047 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3184a9a-9986-41d9-88a7-44b57bcb6ae2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:51:41.895843 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3184a9a-9986-41d9-88a7-44b57bcb6ae2 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:51:41.935929 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1366/2756] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:51:41 2022] POST /identity/v3/projects => generated 373 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:51:41.940387 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353131f7-8eb7-46fd-a9ee-29feb49890ac None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:51:41.952992 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353131f7-8eb7-46fd-a9ee-29feb49890ac None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:51:42.028312 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeb673e-d4c4-4cea-a980-26b07e234319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:51:42.029721 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb673e-d4c4-4cea-a980-26b07e234319 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:42.029928 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb673e-d4c4-4cea-a980-26b07e234319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:42.030580 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb673e-d4c4-4cea-a980-26b07e234319 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:42.031806 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeb673e-d4c4-4cea-a980-26b07e234319 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:51:42.035505 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeb673e-d4c4-4cea-a980-26b07e234319 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:51:42.081675 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1391/2757] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:51:41 2022] POST /identity/v3/users => generated 293 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:51:42.085959 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3184a9a-9986-41d9-88a7-44b57bcb6ae2 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:51:42.097123 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3184a9a-9986-41d9-88a7-44b57bcb6ae2 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:51:42.115490 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c23e1d7-56a1-4cdc-b977-875609e5a53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:51:42.117052 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c23e1d7-56a1-4cdc-b977-875609e5a53f admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:42.117178 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c23e1d7-56a1-4cdc-b977-875609e5a53f admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:42.117378 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c23e1d7-56a1-4cdc-b977-875609e5a53f admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:42.118210 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c23e1d7-56a1-4cdc-b977-875609e5a53f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:51:42.120021 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c23e1d7-56a1-4cdc-b977-875609e5a53f admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:51:42.132884 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1367/2758] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:51:42 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:51:42.137998 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeb673e-d4c4-4cea-a980-26b07e234319 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:51:42.153640 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeb673e-d4c4-4cea-a980-26b07e234319 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:51:42.160880 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1612d4d6-50e4-469e-8871-ad3761e7096d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:51:42.162600 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1612d4d6-50e4-469e-8871-ad3761e7096d admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:42.163185 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1612d4d6-50e4-469e-8871-ad3761e7096d admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:42.163698 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1612d4d6-50e4-469e-8871-ad3761e7096d admin admin] PATH_INFO: `/v3/projects/c52ddf5ec3544b198de9b02b2cc49b03/users/1d6a85a232f047a69c6bdf5a19467988/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:42.195613 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1612d4d6-50e4-469e-8871-ad3761e7096d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c52ddf5ec3544b198de9b02b2cc49b03, user_id=1d6a85a232f047a69c6bdf5a19467988, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:51:42.201088 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1612d4d6-50e4-469e-8871-ad3761e7096d admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:51:42.220680 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1392/2759] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:51:42 2022] PUT /identity/v3/projects/c52ddf5ec3544b198de9b02b2cc49b03/users/1d6a85a232f047a69c6bdf5a19467988/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:51:42.225073 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c23e1d7-56a1-4cdc-b977-875609e5a53f admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:51:42.236431 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c23e1d7-56a1-4cdc-b977-875609e5a53f admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:51:42.313330 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b9de9a-0c72-400f-b346-2297da8ee756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:51:42.314056 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9de9a-0c72-400f-b346-2297da8ee756 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:42.314341 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9de9a-0c72-400f-b346-2297da8ee756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:42.314624 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9de9a-0c72-400f-b346-2297da8ee756 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:42.315388 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b9de9a-0c72-400f-b346-2297da8ee756 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:51:42.320300 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b9de9a-0c72-400f-b346-2297da8ee756 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:51:42.338287 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1368/2760] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:51:42 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:51:42.343186 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1612d4d6-50e4-469e-8871-ad3761e7096d admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:51:42.364003 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1612d4d6-50e4-469e-8871-ad3761e7096d admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:51:42.374190 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c6f812-eff2-40be-ba90-27ab1d8685f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:51:42.375039 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6f812-eff2-40be-ba90-27ab1d8685f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:42.375320 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6f812-eff2-40be-ba90-27ab1d8685f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:42.375594 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6f812-eff2-40be-ba90-27ab1d8685f4 admin admin] PATH_INFO: `/v3/projects/c52ddf5ec3544b198de9b02b2cc49b03/users/1d6a85a232f047a69c6bdf5a19467988/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:42.377858 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c6f812-eff2-40be-ba90-27ab1d8685f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c52ddf5ec3544b198de9b02b2cc49b03, user_id=1d6a85a232f047a69c6bdf5a19467988, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:51:42.381081 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c6f812-eff2-40be-ba90-27ab1d8685f4 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:51:42.399897 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1393/2761] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:51:42 2022] PUT /identity/v3/projects/c52ddf5ec3544b198de9b02b2cc49b03/users/1d6a85a232f047a69c6bdf5a19467988/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:51:47.391160 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd94d54-1903-4678-99c5-7c008af7bb34 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:47.391728 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd94d54-1903-4678-99c5-7c008af7bb34 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:47.391986 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd94d54-1903-4678-99c5-7c008af7bb34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:47.421821 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-fcd94d54-1903-4678-99c5-7c008af7bb34 None None] MFA Rules not processed for user `1d6a85a232f047a69c6bdf5a19467988`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:51:47.423251 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-fcd94d54-1903-4678-99c5-7c008af7bb34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:51:47.464902 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:51:47.497890 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1369/2762] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:51:47 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:51:47.514477 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c6f812-eff2-40be-ba90-27ab1d8685f4 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:51:47.517455 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-a1c6f812-eff2-40be-ba90-27ab1d8685f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:51:47.548030 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:51:47.618245 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c6f812-eff2-40be-ba90-27ab1d8685f4 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:51:47.625365 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4472ff-1035-41d4-b4ee-aac245260424 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74bb1bc6821b4500b26f160ccae5b7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:51:47.626207 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4472ff-1035-41d4-b4ee-aac245260424 service nova] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:47.626959 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4472ff-1035-41d4-b4ee-aac245260424 service nova] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:47.627107 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4472ff-1035-41d4-b4ee-aac245260424 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:47.713357 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4472ff-1035-41d4-b4ee-aac245260424 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:51:47.715016 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4472ff-1035-41d4-b4ee-aac245260424 service nova] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:51:47.724803 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1394/2763] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:51:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:51:48.813542 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26989dc-f364-49eb-a139-5796f72bd770 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:48.814319 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26989dc-f364-49eb-a139-5796f72bd770 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:48.814319 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26989dc-f364-49eb-a139-5796f72bd770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:48.833566 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-d26989dc-f364-49eb-a139-5796f72bd770 None None] MFA Rules not processed for user `038c41319feb49a486832af853c332b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:51:48.835703 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d26989dc-f364-49eb-a139-5796f72bd770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:51:48.843567 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1370/2764] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:51:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:51:57.720194 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=45 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:51:58.827435 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=15 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:52:04.722022 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4472ff-1035-41d4-b4ee-aac245260424 service nova] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:04.724219 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-dc4472ff-1035-41d4-b4ee-aac245260424 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:04.747873 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4472ff-1035-41d4-b4ee-aac245260424 service nova] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:04.759996 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5c0686-cbae-4874-9ee6-4853bf6d8a57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74bb1bc6821b4500b26f160ccae5b7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:04.761015 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c0686-cbae-4874-9ee6-4853bf6d8a57 service nova] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:04.761304 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c0686-cbae-4874-9ee6-4853bf6d8a57 service nova] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:04.761590 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c0686-cbae-4874-9ee6-4853bf6d8a57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:04.763568 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-cd5c0686-cbae-4874-9ee6-4853bf6d8a57 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:04.780064 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5c0686-cbae-4874-9ee6-4853bf6d8a57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:04.782020 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5c0686-cbae-4874-9ee6-4853bf6d8a57 service nova] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:04.796195 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1395/2765] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:52:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:52:07.219632 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26989dc-f364-49eb-a139-5796f72bd770 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:07.222843 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d26989dc-f364-49eb-a139-5796f72bd770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:07.322344 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26989dc-f364-49eb-a139-5796f72bd770 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:07.333407 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f58202-b7fa-4934-a57e-0ec0b377307b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39cc268bad604dd08594e3b39c7ecf5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:07.334273 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f58202-b7fa-4934-a57e-0ec0b377307b service placement] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:07.334562 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f58202-b7fa-4934-a57e-0ec0b377307b service placement] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:07.334852 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f58202-b7fa-4934-a57e-0ec0b377307b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:07.337526 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-89f58202-b7fa-4934-a57e-0ec0b377307b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:07.357352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f58202-b7fa-4934-a57e-0ec0b377307b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:07.357352 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f58202-b7fa-4934-a57e-0ec0b377307b service placement] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:07.377896 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1371/2766] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:52:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:52:09.806303 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5c0686-cbae-4874-9ee6-4853bf6d8a57 service nova] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:09.865191 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5c0686-cbae-4874-9ee6-4853bf6d8a57 service nova] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:09.870389 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeba0137-3b96-4a1f-b619-486fc4a80cf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:09.871054 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeba0137-3b96-4a1f-b619-486fc4a80cf9 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:09.871516 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeba0137-3b96-4a1f-b619-486fc4a80cf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:09.871806 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeba0137-3b96-4a1f-b619-486fc4a80cf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:09.873620 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-aeba0137-3b96-4a1f-b619-486fc4a80cf9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:09.939853 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeba0137-3b96-4a1f-b619-486fc4a80cf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:09.940982 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeba0137-3b96-4a1f-b619-486fc4a80cf9 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:09.951740 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1396/2767] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:52:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:52:14.109310 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f58202-b7fa-4934-a57e-0ec0b377307b service placement] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:14.111016 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-89f58202-b7fa-4934-a57e-0ec0b377307b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:14.126433 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f58202-b7fa-4934-a57e-0ec0b377307b service placement] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:14.132694 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9947f57-1aab-49b6-b464-e1a65a1c5ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:14.134348 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9947f57-1aab-49b6-b464-e1a65a1c5ca3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:14.134721 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9947f57-1aab-49b6-b464-e1a65a1c5ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:14.135003 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9947f57-1aab-49b6-b464-e1a65a1c5ca3 admin admin] PATH_INFO: `/v3/users/8d15a76041274e7381087bd21eb674c9` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:14.136543 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9947f57-1aab-49b6-b464-e1a65a1c5ca3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d15a76041274e7381087bd21eb674c9)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:14.138726 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9947f57-1aab-49b6-b464-e1a65a1c5ca3 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:14.199985 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c9947f57-1aab-49b6-b464-e1a65a1c5ca3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d15a76041274e7381087bd21eb674c9'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:14.219259 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c9947f57-1aab-49b6-b464-e1a65a1c5ca3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d15a76041274e7381087bd21eb674c9'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:14.220134 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c9947f57-1aab-49b6-b464-e1a65a1c5ca3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d15a76041274e7381087bd21eb674c9'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:14.227559 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c9947f57-1aab-49b6-b464-e1a65a1c5ca3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d15a76041274e7381087bd21eb674c9'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:14.228291 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c9947f57-1aab-49b6-b464-e1a65a1c5ca3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d15a76041274e7381087bd21eb674c9'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:14.245885 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1372/2768] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:52:14 2022] DELETE /identity/v3/users/8d15a76041274e7381087bd21eb674c9 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:52:14.442381 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeba0137-3b96-4a1f-b619-486fc4a80cf9 service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:14.453999 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeba0137-3b96-4a1f-b619-486fc4a80cf9 service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:14.530968 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30f34e4-b4c7-43c8-a1d5-dffcc12d66a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:14.531951 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f34e4-b4c7-43c8-a1d5-dffcc12d66a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:14.532240 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f34e4-b4c7-43c8-a1d5-dffcc12d66a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:14.532517 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f34e4-b4c7-43c8-a1d5-dffcc12d66a2 admin admin] PATH_INFO: `/v3/projects/7787ee132e9e4e23b1407718fb366406` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:14.533826 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30f34e4-b4c7-43c8-a1d5-dffcc12d66a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7787ee132e9e4e23b1407718fb366406)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:14.535792 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30f34e4-b4c7-43c8-a1d5-dffcc12d66a2 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:14.586629 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:52:14.595263 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-e30f34e4-b4c7-43c8-a1d5-dffcc12d66a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7787ee132e9e4e23b1407718fb366406', 'request_id': 'req-e30f34e4-b4c7-43c8-a1d5-dffcc12d66a2'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:14.602660 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-e30f34e4-b4c7-43c8-a1d5-dffcc12d66a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7787ee132e9e4e23b1407718fb366406', 'request_id': 'req-e30f34e4-b4c7-43c8-a1d5-dffcc12d66a2'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:14.608889 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-e30f34e4-b4c7-43c8-a1d5-dffcc12d66a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7787ee132e9e4e23b1407718fb366406', 'request_id': 'req-e30f34e4-b4c7-43c8-a1d5-dffcc12d66a2'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:14.624106 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-e30f34e4-b4c7-43c8-a1d5-dffcc12d66a2 admin admin] The token cache is being invalidate because project 7787ee132e9e4e23b1407718fb366406 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:52:14.624398 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-e30f34e4-b4c7-43c8-a1d5-dffcc12d66a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:14.625080 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-e30f34e4-b4c7-43c8-a1d5-dffcc12d66a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:14.626702 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1397/2769] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:52:14 2022] DELETE /identity/v3/projects/7787ee132e9e4e23b1407718fb366406 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:52:14.632139 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a7b4db-497b-4656-9a7a-a748426f8ed5 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:14.633238 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a7b4db-497b-4656-9a7a-a748426f8ed5 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:14.633870 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a7b4db-497b-4656-9a7a-a748426f8ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:14.656447 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-f8a7b4db-497b-4656-9a7a-a748426f8ed5 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:52:14.657712 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-f8a7b4db-497b-4656-9a7a-a748426f8ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:14.729206 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1373/2770] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:52:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:52:14.737425 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc4ed8e-e03e-4644-a3c5-68f6373c1c53 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:14.737812 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc4ed8e-e03e-4644-a3c5-68f6373c1c53 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:14.738164 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc4ed8e-e03e-4644-a3c5-68f6373c1c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:14.762378 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-ccc4ed8e-e03e-4644-a3c5-68f6373c1c53 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:52:14.763738 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-ccc4ed8e-e03e-4644-a3c5-68f6373c1c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:14.773760 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1398/2771] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:52:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:52:14.778368 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a7b4db-497b-4656-9a7a-a748426f8ed5 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:14.783595 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:52:14.790580 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a7b4db-497b-4656-9a7a-a748426f8ed5 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:14.798811 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1c78b3-8fe0-4955-bf5a-2cf9d739b871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:14.799639 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1c78b3-8fe0-4955-bf5a-2cf9d739b871 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:14.800255 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1c78b3-8fe0-4955-bf5a-2cf9d739b871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:14.800602 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1c78b3-8fe0-4955-bf5a-2cf9d739b871 admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:14.801749 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1c78b3-8fe0-4955-bf5a-2cf9d739b871 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:52:14.802558 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1c78b3-8fe0-4955-bf5a-2cf9d739b871 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:14.805104 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1c78b3-8fe0-4955-bf5a-2cf9d739b871 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:14.822848 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1374/2772] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:52:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:52:14.824362 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc4ed8e-e03e-4644-a3c5-68f6373c1c53 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:14.839425 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc4ed8e-e03e-4644-a3c5-68f6373c1c53 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:14.844680 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba513b2e-e869-4d26-a5b8-324e0f05f288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:14.845355 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba513b2e-e869-4d26-a5b8-324e0f05f288 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:14.845644 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba513b2e-e869-4d26-a5b8-324e0f05f288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:14.845953 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba513b2e-e869-4d26-a5b8-324e0f05f288 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:14.846671 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba513b2e-e869-4d26-a5b8-324e0f05f288 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:14.848633 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba513b2e-e869-4d26-a5b8-324e0f05f288 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:14.879293 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1399/2773] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:52:14 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:52:14.880708 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1c78b3-8fe0-4955-bf5a-2cf9d739b871 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:14.892348 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1c78b3-8fe0-4955-bf5a-2cf9d739b871 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:14.954228 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab78ccf-a439-4c26-bfab-f64e54081448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:14.954903 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab78ccf-a439-4c26-bfab-f64e54081448 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:14.955207 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab78ccf-a439-4c26-bfab-f64e54081448 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:14.955492 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab78ccf-a439-4c26-bfab-f64e54081448 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:14.956210 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab78ccf-a439-4c26-bfab-f64e54081448 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:14.958191 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab78ccf-a439-4c26-bfab-f64e54081448 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:14.983179 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1375/2774] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:52:14 2022] POST /identity/v3/users => generated 287 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:52:14.989846 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba513b2e-e869-4d26-a5b8-324e0f05f288 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:15.001491 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba513b2e-e869-4d26-a5b8-324e0f05f288 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:15.006505 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a45b08-1b22-44a3-92ee-d2786211a8c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:15.007909 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a45b08-1b22-44a3-92ee-d2786211a8c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:15.008212 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a45b08-1b22-44a3-92ee-d2786211a8c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:15.008493 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a45b08-1b22-44a3-92ee-d2786211a8c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:15.009248 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a45b08-1b22-44a3-92ee-d2786211a8c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:15.010898 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a45b08-1b22-44a3-92ee-d2786211a8c2 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:15.024533 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1400/2775] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:52:15.028930 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab78ccf-a439-4c26-bfab-f64e54081448 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:15.041676 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab78ccf-a439-4c26-bfab-f64e54081448 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:15.046928 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de26f1b2-5377-4796-8f75-59ee348f2037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:15.047651 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de26f1b2-5377-4796-8f75-59ee348f2037 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:15.047939 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de26f1b2-5377-4796-8f75-59ee348f2037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:15.048219 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de26f1b2-5377-4796-8f75-59ee348f2037 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:15.049002 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de26f1b2-5377-4796-8f75-59ee348f2037 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:15.050816 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de26f1b2-5377-4796-8f75-59ee348f2037 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:15.068048 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1376/2776] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:52:15.069322 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a45b08-1b22-44a3-92ee-d2786211a8c2 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:15.081195 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a45b08-1b22-44a3-92ee-d2786211a8c2 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:15.087829 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6ff955-9f4d-4764-9158-414d67ffdcf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:15.088938 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ff955-9f4d-4764-9158-414d67ffdcf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:15.089352 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ff955-9f4d-4764-9158-414d67ffdcf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:15.089636 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ff955-9f4d-4764-9158-414d67ffdcf4 admin admin] PATH_INFO: `/v3/projects/3e815c8ff8484929bff5d880112bf9a9/users/d76c7d5df9ce493ca58f2cd0ccacaf67/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:15.110839 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6ff955-9f4d-4764-9158-414d67ffdcf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e815c8ff8484929bff5d880112bf9a9, user_id=d76c7d5df9ce493ca58f2cd0ccacaf67, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:15.113715 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6ff955-9f4d-4764-9158-414d67ffdcf4 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:15.130438 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1401/2777] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:52:15 2022] PUT /identity/v3/projects/3e815c8ff8484929bff5d880112bf9a9/users/d76c7d5df9ce493ca58f2cd0ccacaf67/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:52:15.141571 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ead4264-26e0-430e-81cd-ae0caad1c2c7 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:15.142221 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ead4264-26e0-430e-81cd-ae0caad1c2c7 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:15.142221 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ead4264-26e0-430e-81cd-ae0caad1c2c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:15.172614 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-3ead4264-26e0-430e-81cd-ae0caad1c2c7 None None] MFA Rules not processed for user `d76c7d5df9ce493ca58f2cd0ccacaf67`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:52:15.174021 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-3ead4264-26e0-430e-81cd-ae0caad1c2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:15.253082 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1377/2778] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:52:15 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:52:15.258505 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6ff955-9f4d-4764-9158-414d67ffdcf4 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:15.329961 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6ff955-9f4d-4764-9158-414d67ffdcf4 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:15.334979 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f77561-c6b1-45f5-aa9a-0fbcd63f6ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:15.336453 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f77561-c6b1-45f5-aa9a-0fbcd63f6ebb admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:15.336737 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f77561-c6b1-45f5-aa9a-0fbcd63f6ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:15.337005 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f77561-c6b1-45f5-aa9a-0fbcd63f6ebb admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:15.337746 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f77561-c6b1-45f5-aa9a-0fbcd63f6ebb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:15.339683 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f77561-c6b1-45f5-aa9a-0fbcd63f6ebb admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:15.367470 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1402/2779] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:52:15 2022] POST /identity/v3/projects => generated 373 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:52:15.374420 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ead4264-26e0-430e-81cd-ae0caad1c2c7 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:15.385976 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ead4264-26e0-430e-81cd-ae0caad1c2c7 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:15.438568 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:52:15.446402 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e092980-e443-49c9-9d85-cbf7687311f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:15.447340 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e092980-e443-49c9-9d85-cbf7687311f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:15.447846 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e092980-e443-49c9-9d85-cbf7687311f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:15.448126 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e092980-e443-49c9-9d85-cbf7687311f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:15.448825 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e092980-e443-49c9-9d85-cbf7687311f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:15.450840 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e092980-e443-49c9-9d85-cbf7687311f8 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:15.471699 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1378/2780] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:52:15 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:52:15.476179 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f77561-c6b1-45f5-aa9a-0fbcd63f6ebb admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:15.487414 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f77561-c6b1-45f5-aa9a-0fbcd63f6ebb admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:15.492190 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25244d4-4fe9-41d1-b7cf-672c04df6cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:15.492858 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25244d4-4fe9-41d1-b7cf-672c04df6cab admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:15.493148 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25244d4-4fe9-41d1-b7cf-672c04df6cab admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:15.493428 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25244d4-4fe9-41d1-b7cf-672c04df6cab admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:15.494202 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25244d4-4fe9-41d1-b7cf-672c04df6cab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:15.495809 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25244d4-4fe9-41d1-b7cf-672c04df6cab admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:15.507617 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1403/2781] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:52:15.511882 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e092980-e443-49c9-9d85-cbf7687311f8 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:15.523313 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e092980-e443-49c9-9d85-cbf7687311f8 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:15.528074 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7badca69-3dcf-4e12-8c4d-1d38a46e7855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:15.528968 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7badca69-3dcf-4e12-8c4d-1d38a46e7855 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:15.529248 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7badca69-3dcf-4e12-8c4d-1d38a46e7855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:15.529519 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7badca69-3dcf-4e12-8c4d-1d38a46e7855 admin admin] PATH_INFO: `/v3/projects/23258b98d5d64d1eac8077deff8dfd27/users/7777f32bff0c4765b5aabb11dfd92d61/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:15.545815 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7badca69-3dcf-4e12-8c4d-1d38a46e7855 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23258b98d5d64d1eac8077deff8dfd27, user_id=7777f32bff0c4765b5aabb11dfd92d61, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:15.548486 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7badca69-3dcf-4e12-8c4d-1d38a46e7855 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:15.561793 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1379/2782] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:52:15 2022] PUT /identity/v3/projects/23258b98d5d64d1eac8077deff8dfd27/users/7777f32bff0c4765b5aabb11dfd92d61/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:52:15.565804 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25244d4-4fe9-41d1-b7cf-672c04df6cab admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:15.576770 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25244d4-4fe9-41d1-b7cf-672c04df6cab admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:15.611901 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:52:15.638995 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f666f8d-a930-4f03-887d-b30a7bdbad9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:15.639668 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f666f8d-a930-4f03-887d-b30a7bdbad9f admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:15.639966 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f666f8d-a930-4f03-887d-b30a7bdbad9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:15.640378 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f666f8d-a930-4f03-887d-b30a7bdbad9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:15.641118 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f666f8d-a930-4f03-887d-b30a7bdbad9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:15.643124 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f666f8d-a930-4f03-887d-b30a7bdbad9f admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:15.654341 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1404/2783] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:52:15.658746 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7badca69-3dcf-4e12-8c4d-1d38a46e7855 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:15.669262 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7badca69-3dcf-4e12-8c4d-1d38a46e7855 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:15.673868 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c48c104-b356-4dd6-b826-12e76211cc86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:15.674752 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c48c104-b356-4dd6-b826-12e76211cc86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:15.675032 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c48c104-b356-4dd6-b826-12e76211cc86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:15.675322 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c48c104-b356-4dd6-b826-12e76211cc86 admin admin] PATH_INFO: `/v3/projects/23258b98d5d64d1eac8077deff8dfd27/users/7777f32bff0c4765b5aabb11dfd92d61/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:15.677247 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c48c104-b356-4dd6-b826-12e76211cc86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23258b98d5d64d1eac8077deff8dfd27, user_id=7777f32bff0c4765b5aabb11dfd92d61, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:15.680221 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c48c104-b356-4dd6-b826-12e76211cc86 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:15.690810 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1380/2784] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:52:15 2022] PUT /identity/v3/projects/23258b98d5d64d1eac8077deff8dfd27/users/7777f32bff0c4765b5aabb11dfd92d61/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:52:15.703145 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56685a68-cb4e-41ca-b320-5371b1252fdd None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:15.703443 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56685a68-cb4e-41ca-b320-5371b1252fdd None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:15.703725 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56685a68-cb4e-41ca-b320-5371b1252fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:15.733059 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-56685a68-cb4e-41ca-b320-5371b1252fdd None None] MFA Rules not processed for user `7777f32bff0c4765b5aabb11dfd92d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:52:15.734314 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-56685a68-cb4e-41ca-b320-5371b1252fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:15.819647 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1405/2785] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:52:15 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:52:15.831093 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c48c104-b356-4dd6-b826-12e76211cc86 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:15.832825 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-9c48c104-b356-4dd6-b826-12e76211cc86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:15.902661 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c48c104-b356-4dd6-b826-12e76211cc86 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:15.906660 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336e3527-4dab-45da-8900-f167879c1771 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:15.907387 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e3527-4dab-45da-8900-f167879c1771 service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:15.907741 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e3527-4dab-45da-8900-f167879c1771 service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:15.908082 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e3527-4dab-45da-8900-f167879c1771 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:15.921588 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336e3527-4dab-45da-8900-f167879c1771 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:15.923196 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336e3527-4dab-45da-8900-f167879c1771 service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:15.935784 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1381/2786] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:52:15 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:52:16.373260 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56685a68-cb4e-41ca-b320-5371b1252fdd None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:16.436469 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56685a68-cb4e-41ca-b320-5371b1252fdd None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:16.440978 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7accac34-2550-42f5-ad7b-70c6513a8aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:16.441911 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7accac34-2550-42f5-ad7b-70c6513a8aeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:16.442191 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7accac34-2550-42f5-ad7b-70c6513a8aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:16.442465 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7accac34-2550-42f5-ad7b-70c6513a8aeb admin admin] PATH_INFO: `/v3/users/d76c7d5df9ce493ca58f2cd0ccacaf67` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:16.443648 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7accac34-2550-42f5-ad7b-70c6513a8aeb admin admin] RBAC: Authorizing `identity:delete_user(user_id=d76c7d5df9ce493ca58f2cd0ccacaf67)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:16.446109 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7accac34-2550-42f5-ad7b-70c6513a8aeb admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:16.495422 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7accac34-2550-42f5-ad7b-70c6513a8aeb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd76c7d5df9ce493ca58f2cd0ccacaf67'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:16.505840 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:52:16.509089 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7accac34-2550-42f5-ad7b-70c6513a8aeb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd76c7d5df9ce493ca58f2cd0ccacaf67'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:16.516730 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7accac34-2550-42f5-ad7b-70c6513a8aeb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd76c7d5df9ce493ca58f2cd0ccacaf67'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:16.517342 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7accac34-2550-42f5-ad7b-70c6513a8aeb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd76c7d5df9ce493ca58f2cd0ccacaf67'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:16.517978 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7accac34-2550-42f5-ad7b-70c6513a8aeb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd76c7d5df9ce493ca58f2cd0ccacaf67'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:16.531040 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1406/2787] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:52:16 2022] DELETE /identity/v3/users/d76c7d5df9ce493ca58f2cd0ccacaf67 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:52:16.535695 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c18117-1ebe-4e5d-bdae-9c86fb9eda37 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:16.536332 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c18117-1ebe-4e5d-bdae-9c86fb9eda37 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:16.536332 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c18117-1ebe-4e5d-bdae-9c86fb9eda37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:16.555141 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-26c18117-1ebe-4e5d-bdae-9c86fb9eda37 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:52:16.556365 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-26c18117-1ebe-4e5d-bdae-9c86fb9eda37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:16.616766 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1382/2788] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:52:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:52:16.628426 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7accac34-2550-42f5-ad7b-70c6513a8aeb admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:16.630633 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-7accac34-2550-42f5-ad7b-70c6513a8aeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:16.696098 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7accac34-2550-42f5-ad7b-70c6513a8aeb admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:16.699887 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddaf479-470b-498c-877e-d301d0b2557b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:16.700566 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddaf479-470b-498c-877e-d301d0b2557b service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:16.700912 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddaf479-470b-498c-877e-d301d0b2557b service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:16.701090 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddaf479-470b-498c-877e-d301d0b2557b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:16.710829 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddaf479-470b-498c-877e-d301d0b2557b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:16.712268 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddaf479-470b-498c-877e-d301d0b2557b service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:16.722211 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1407/2789] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:52:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:52:17.000469 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c18117-1ebe-4e5d-bdae-9c86fb9eda37 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:17.002367 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-26c18117-1ebe-4e5d-bdae-9c86fb9eda37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:17.018178 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c18117-1ebe-4e5d-bdae-9c86fb9eda37 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:17.024158 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1a7d52-cd0d-4565-a82a-895f95692772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:17.025773 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a7d52-cd0d-4565-a82a-895f95692772 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:17.025773 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a7d52-cd0d-4565-a82a-895f95692772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:17.025773 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a7d52-cd0d-4565-a82a-895f95692772 admin admin] PATH_INFO: `/v3/projects/3e815c8ff8484929bff5d880112bf9a9` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:17.027517 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1a7d52-cd0d-4565-a82a-895f95692772 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e815c8ff8484929bff5d880112bf9a9)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:17.029996 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1a7d52-cd0d-4565-a82a-895f95692772 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:17.091651 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:52:17.093930 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7b1a7d52-cd0d-4565-a82a-895f95692772 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e815c8ff8484929bff5d880112bf9a9', 'request_id': 'req-7b1a7d52-cd0d-4565-a82a-895f95692772'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:17.104641 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7b1a7d52-cd0d-4565-a82a-895f95692772 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e815c8ff8484929bff5d880112bf9a9', 'request_id': 'req-7b1a7d52-cd0d-4565-a82a-895f95692772'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:17.114539 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7b1a7d52-cd0d-4565-a82a-895f95692772 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e815c8ff8484929bff5d880112bf9a9', 'request_id': 'req-7b1a7d52-cd0d-4565-a82a-895f95692772'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:17.131196 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7b1a7d52-cd0d-4565-a82a-895f95692772 admin admin] The token cache is being invalidate because project 3e815c8ff8484929bff5d880112bf9a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:52:17.131506 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7b1a7d52-cd0d-4565-a82a-895f95692772 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:17.132190 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7b1a7d52-cd0d-4565-a82a-895f95692772 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:17.134076 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1383/2790] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:52:16 2022] DELETE /identity/v3/projects/3e815c8ff8484929bff5d880112bf9a9 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:52:17.137936 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddaf479-470b-498c-877e-d301d0b2557b service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:17.148871 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddaf479-470b-498c-877e-d301d0b2557b service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:17.209530 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd07cb0-0bb0-4eca-9087-5e23cac51ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:17.210448 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd07cb0-0bb0-4eca-9087-5e23cac51ffe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:17.210735 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd07cb0-0bb0-4eca-9087-5e23cac51ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:17.211083 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd07cb0-0bb0-4eca-9087-5e23cac51ffe admin admin] PATH_INFO: `/v3/users/7777f32bff0c4765b5aabb11dfd92d61` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:17.212328 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd07cb0-0bb0-4eca-9087-5e23cac51ffe admin admin] RBAC: Authorizing `identity:delete_user(user_id=7777f32bff0c4765b5aabb11dfd92d61)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:17.214293 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd07cb0-0bb0-4eca-9087-5e23cac51ffe admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:17.271281 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0fd07cb0-0bb0-4eca-9087-5e23cac51ffe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7777f32bff0c4765b5aabb11dfd92d61'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:17.284320 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0fd07cb0-0bb0-4eca-9087-5e23cac51ffe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7777f32bff0c4765b5aabb11dfd92d61'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:17.292988 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0fd07cb0-0bb0-4eca-9087-5e23cac51ffe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7777f32bff0c4765b5aabb11dfd92d61'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:17.293599 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0fd07cb0-0bb0-4eca-9087-5e23cac51ffe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7777f32bff0c4765b5aabb11dfd92d61'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:17.294255 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-0fd07cb0-0bb0-4eca-9087-5e23cac51ffe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7777f32bff0c4765b5aabb11dfd92d61'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:17.302665 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:52:17.308586 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1408/2791] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:52:17 2022] DELETE /identity/v3/users/7777f32bff0c4765b5aabb11dfd92d61 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:52:17.586412 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1a7d52-cd0d-4565-a82a-895f95692772 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:17.587897 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-7b1a7d52-cd0d-4565-a82a-895f95692772 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:17.602518 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1a7d52-cd0d-4565-a82a-895f95692772 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:17.662421 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0222c15c-9f4c-413c-a87c-664ee41648df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:17.663766 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222c15c-9f4c-413c-a87c-664ee41648df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:17.664728 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222c15c-9f4c-413c-a87c-664ee41648df admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:17.665010 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222c15c-9f4c-413c-a87c-664ee41648df admin admin] PATH_INFO: `/v3/projects/23258b98d5d64d1eac8077deff8dfd27` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:17.666356 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0222c15c-9f4c-413c-a87c-664ee41648df admin admin] RBAC: Authorizing `identity:delete_project(project_id=23258b98d5d64d1eac8077deff8dfd27)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:17.668280 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0222c15c-9f4c-413c-a87c-664ee41648df admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:17.723349 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-0222c15c-9f4c-413c-a87c-664ee41648df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23258b98d5d64d1eac8077deff8dfd27', 'request_id': 'req-0222c15c-9f4c-413c-a87c-664ee41648df'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:17.730636 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-0222c15c-9f4c-413c-a87c-664ee41648df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23258b98d5d64d1eac8077deff8dfd27', 'request_id': 'req-0222c15c-9f4c-413c-a87c-664ee41648df'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:17.736167 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-0222c15c-9f4c-413c-a87c-664ee41648df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23258b98d5d64d1eac8077deff8dfd27', 'request_id': 'req-0222c15c-9f4c-413c-a87c-664ee41648df'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:17.750353 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-0222c15c-9f4c-413c-a87c-664ee41648df admin admin] The token cache is being invalidate because project 23258b98d5d64d1eac8077deff8dfd27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:52:17.750635 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-0222c15c-9f4c-413c-a87c-664ee41648df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:17.751300 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-0222c15c-9f4c-413c-a87c-664ee41648df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:17.752847 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1384/2792] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:52:17 2022] DELETE /identity/v3/projects/23258b98d5d64d1eac8077deff8dfd27 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:52:17.758589 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70819764-7124-4cc2-82c7-d83346728ea4 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:17.759083 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70819764-7124-4cc2-82c7-d83346728ea4 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:17.759182 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70819764-7124-4cc2-82c7-d83346728ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:17.778361 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-70819764-7124-4cc2-82c7-d83346728ea4 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:52:17.779578 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-70819764-7124-4cc2-82c7-d83346728ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:17.840241 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1409/2793] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:52:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:52:17.847978 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb05d0-8696-4f66-abab-5c344281bbe5 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:17.848266 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb05d0-8696-4f66-abab-5c344281bbe5 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:17.848544 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb05d0-8696-4f66-abab-5c344281bbe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:17.867196 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-0dbb05d0-8696-4f66-abab-5c344281bbe5 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:52:17.868407 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-0dbb05d0-8696-4f66-abab-5c344281bbe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:17.876344 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1385/2794] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:52:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:52:17.880831 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70819764-7124-4cc2-82c7-d83346728ea4 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:17.891480 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70819764-7124-4cc2-82c7-d83346728ea4 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:17.896080 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293e02d4-7e3d-4b92-ac9f-0de804080004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:17.896734 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e02d4-7e3d-4b92-ac9f-0de804080004 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:17.897019 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e02d4-7e3d-4b92-ac9f-0de804080004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:17.897297 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e02d4-7e3d-4b92-ac9f-0de804080004 admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:17.898969 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293e02d4-7e3d-4b92-ac9f-0de804080004 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:52:17.899275 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293e02d4-7e3d-4b92-ac9f-0de804080004 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:17.900877 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293e02d4-7e3d-4b92-ac9f-0de804080004 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:17.912657 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1410/2795] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:52:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:52:17.916985 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbb05d0-8696-4f66-abab-5c344281bbe5 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:17.926299 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbb05d0-8696-4f66-abab-5c344281bbe5 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:17.931090 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656fc934-988f-4251-b9cb-f68e132e3be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:17.931738 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656fc934-988f-4251-b9cb-f68e132e3be5 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:17.932018 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656fc934-988f-4251-b9cb-f68e132e3be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:17.932290 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656fc934-988f-4251-b9cb-f68e132e3be5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:17.933036 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656fc934-988f-4251-b9cb-f68e132e3be5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:17.934958 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656fc934-988f-4251-b9cb-f68e132e3be5 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:17.961654 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1386/2796] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:52:17 2022] POST /identity/v3/projects => generated 363 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:52:17.966056 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293e02d4-7e3d-4b92-ac9f-0de804080004 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:17.977309 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293e02d4-7e3d-4b92-ac9f-0de804080004 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:18.037531 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4e833f-5ddd-42b0-9aa0-6367d6739f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:18.038890 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e833f-5ddd-42b0-9aa0-6367d6739f80 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:18.039241 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e833f-5ddd-42b0-9aa0-6367d6739f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:18.039610 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e833f-5ddd-42b0-9aa0-6367d6739f80 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:18.040216 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4e833f-5ddd-42b0-9aa0-6367d6739f80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:18.042195 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4e833f-5ddd-42b0-9aa0-6367d6739f80 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:18.062834 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1411/2797] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:52:17 2022] POST /identity/v3/users => generated 282 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:52:18.067190 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656fc934-988f-4251-b9cb-f68e132e3be5 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:18.077657 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656fc934-988f-4251-b9cb-f68e132e3be5 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:18.083548 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c6100-683b-433f-8cf4-0e8990a4f61b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:18.084338 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c6100-683b-433f-8cf4-0e8990a4f61b admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:18.084623 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c6100-683b-433f-8cf4-0e8990a4f61b admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:18.084898 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c6100-683b-433f-8cf4-0e8990a4f61b admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:18.085642 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48c6100-683b-433f-8cf4-0e8990a4f61b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:18.087273 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48c6100-683b-433f-8cf4-0e8990a4f61b admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:18.097902 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1387/2798] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:52:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:52:18.102628 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4e833f-5ddd-42b0-9aa0-6367d6739f80 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:18.112763 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4e833f-5ddd-42b0-9aa0-6367d6739f80 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:18.118616 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c708527-8ceb-4064-97b6-cc3514a9a61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:18.119318 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c708527-8ceb-4064-97b6-cc3514a9a61f admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:18.119599 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c708527-8ceb-4064-97b6-cc3514a9a61f admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:18.119871 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c708527-8ceb-4064-97b6-cc3514a9a61f admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:18.120603 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c708527-8ceb-4064-97b6-cc3514a9a61f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:18.122242 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c708527-8ceb-4064-97b6-cc3514a9a61f admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:18.133003 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1412/2799] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:52:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:52:18.137153 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c6100-683b-433f-8cf4-0e8990a4f61b admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:18.147232 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c6100-683b-433f-8cf4-0e8990a4f61b admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:18.153006 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c4ea48-16fb-4b6a-9362-8c04c33d0a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:18.153931 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4ea48-16fb-4b6a-9362-8c04c33d0a4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:18.154210 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4ea48-16fb-4b6a-9362-8c04c33d0a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:18.154481 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4ea48-16fb-4b6a-9362-8c04c33d0a4f admin admin] PATH_INFO: `/v3/projects/95a26ac1eb1a4434b42be05c98931c82/users/13d635046ede46f8bb6e8e43ae83b577/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:18.169633 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c4ea48-16fb-4b6a-9362-8c04c33d0a4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=95a26ac1eb1a4434b42be05c98931c82, user_id=13d635046ede46f8bb6e8e43ae83b577, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:18.172336 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c4ea48-16fb-4b6a-9362-8c04c33d0a4f admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:18.186654 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1388/2800] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:52:18 2022] PUT /identity/v3/projects/95a26ac1eb1a4434b42be05c98931c82/users/13d635046ede46f8bb6e8e43ae83b577/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:52:18.197378 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b56057-8b62-4acd-ad95-b7a9fbe17345 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:18.197669 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b56057-8b62-4acd-ad95-b7a9fbe17345 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:18.197989 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b56057-8b62-4acd-ad95-b7a9fbe17345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:18.230725 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-a6b56057-8b62-4acd-ad95-b7a9fbe17345 None None] MFA Rules not processed for user `13d635046ede46f8bb6e8e43ae83b577`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:52:18.232489 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-a6b56057-8b62-4acd-ad95-b7a9fbe17345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:18.295954 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:52:18.307920 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1413/2801] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:52:18 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:52:18.312535 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c4ea48-16fb-4b6a-9362-8c04c33d0a4f admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:18.317430 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:52:18.391217 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c4ea48-16fb-4b6a-9362-8c04c33d0a4f admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:18.396114 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b59f368-2a7e-4393-aa7b-559049de95b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:18.396863 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59f368-2a7e-4393-aa7b-559049de95b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:18.397237 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59f368-2a7e-4393-aa7b-559049de95b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:18.398163 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59f368-2a7e-4393-aa7b-559049de95b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:18.399065 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b59f368-2a7e-4393-aa7b-559049de95b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:18.401198 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b59f368-2a7e-4393-aa7b-559049de95b6 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:18.427996 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1389/2802] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:52:18 2022] POST /identity/v3/projects => generated 363 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:52:18.432820 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b56057-8b62-4acd-ad95-b7a9fbe17345 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:18.443831 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b56057-8b62-4acd-ad95-b7a9fbe17345 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:18.502375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c99832d-d390-4a9a-9d02-384b4701a6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:18.503055 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c99832d-d390-4a9a-9d02-384b4701a6e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:18.503341 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c99832d-d390-4a9a-9d02-384b4701a6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:18.503623 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c99832d-d390-4a9a-9d02-384b4701a6e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:18.504313 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c99832d-d390-4a9a-9d02-384b4701a6e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:18.506512 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c99832d-d390-4a9a-9d02-384b4701a6e0 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:18.526435 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1414/2803] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:52:18 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:52:18.530492 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b59f368-2a7e-4393-aa7b-559049de95b6 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:18.541424 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b59f368-2a7e-4393-aa7b-559049de95b6 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:18.546300 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3061453e-1167-47f7-b97e-7508ec36aa8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:18.546970 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061453e-1167-47f7-b97e-7508ec36aa8f admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:18.547264 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061453e-1167-47f7-b97e-7508ec36aa8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:18.547545 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061453e-1167-47f7-b97e-7508ec36aa8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:18.548282 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3061453e-1167-47f7-b97e-7508ec36aa8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:18.549929 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3061453e-1167-47f7-b97e-7508ec36aa8f admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:18.562018 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1390/2804] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:52:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:52:18.566477 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c99832d-d390-4a9a-9d02-384b4701a6e0 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:18.576687 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c99832d-d390-4a9a-9d02-384b4701a6e0 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:18.581310 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ded08f-daf0-4e2d-881a-d5978bb48284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:18.582221 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ded08f-daf0-4e2d-881a-d5978bb48284 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:18.582506 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ded08f-daf0-4e2d-881a-d5978bb48284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:18.582779 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ded08f-daf0-4e2d-881a-d5978bb48284 admin admin] PATH_INFO: `/v3/projects/6090bd704a8e46d89913ef118d6bdfcd/users/63513152a0104794b2aa724f7c671092/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:18.597196 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ded08f-daf0-4e2d-881a-d5978bb48284 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6090bd704a8e46d89913ef118d6bdfcd, user_id=63513152a0104794b2aa724f7c671092, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:18.599852 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ded08f-daf0-4e2d-881a-d5978bb48284 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:18.613316 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1415/2805] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:52:18 2022] PUT /identity/v3/projects/6090bd704a8e46d89913ef118d6bdfcd/users/63513152a0104794b2aa724f7c671092/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:52:18.617851 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3061453e-1167-47f7-b97e-7508ec36aa8f admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:18.628000 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3061453e-1167-47f7-b97e-7508ec36aa8f admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:18.692165 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e36b6-bd8b-4d7b-ad37-ca43226d8aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:18.692773 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e36b6-bd8b-4d7b-ad37-ca43226d8aec admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:18.693059 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e36b6-bd8b-4d7b-ad37-ca43226d8aec admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:18.693338 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e36b6-bd8b-4d7b-ad37-ca43226d8aec admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:18.694098 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324e36b6-bd8b-4d7b-ad37-ca43226d8aec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:18.695935 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324e36b6-bd8b-4d7b-ad37-ca43226d8aec admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:18.705859 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1391/2806] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:52:18 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:52:18.710015 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ded08f-daf0-4e2d-881a-d5978bb48284 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:18.720430 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ded08f-daf0-4e2d-881a-d5978bb48284 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:18.725026 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b0312c-3fbe-42ee-8287-ecc28612dc51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:18.726495 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0312c-3fbe-42ee-8287-ecc28612dc51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:18.726780 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0312c-3fbe-42ee-8287-ecc28612dc51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:18.727057 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0312c-3fbe-42ee-8287-ecc28612dc51 admin admin] PATH_INFO: `/v3/projects/6090bd704a8e46d89913ef118d6bdfcd/users/63513152a0104794b2aa724f7c671092/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:18.729075 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b0312c-3fbe-42ee-8287-ecc28612dc51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6090bd704a8e46d89913ef118d6bdfcd, user_id=63513152a0104794b2aa724f7c671092, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:18.731747 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b0312c-3fbe-42ee-8287-ecc28612dc51 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:18.743007 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1416/2807] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:52:18 2022] PUT /identity/v3/projects/6090bd704a8e46d89913ef118d6bdfcd/users/63513152a0104794b2aa724f7c671092/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:52:18.754890 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f10fa2-f46c-4238-a657-87cb82b005e8 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:18.755185 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f10fa2-f46c-4238-a657-87cb82b005e8 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:18.755467 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f10fa2-f46c-4238-a657-87cb82b005e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:18.783186 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-32f10fa2-f46c-4238-a657-87cb82b005e8 None None] MFA Rules not processed for user `63513152a0104794b2aa724f7c671092`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:52:18.784395 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-32f10fa2-f46c-4238-a657-87cb82b005e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:18.847933 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:52:18.857656 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1392/2808] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:52:18 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:52:18.866844 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b0312c-3fbe-42ee-8287-ecc28612dc51 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:18.868715 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-e4b0312c-3fbe-42ee-8287-ecc28612dc51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:18.926349 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b0312c-3fbe-42ee-8287-ecc28612dc51 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:18.930564 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfca1b14-c58a-49bd-8357-85d77ff4c595 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:18.931239 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfca1b14-c58a-49bd-8357-85d77ff4c595 service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:18.931507 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfca1b14-c58a-49bd-8357-85d77ff4c595 service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:18.931786 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfca1b14-c58a-49bd-8357-85d77ff4c595 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:18.940988 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfca1b14-c58a-49bd-8357-85d77ff4c595 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:18.942502 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfca1b14-c58a-49bd-8357-85d77ff4c595 service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:18.952670 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1417/2809] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:52:18 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:52:19.368103 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f10fa2-f46c-4238-a657-87cb82b005e8 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:19.432596 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f10fa2-f46c-4238-a657-87cb82b005e8 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:19.437112 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8c913e-f41e-4a31-98f6-7f8e297a29d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:19.438029 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8c913e-f41e-4a31-98f6-7f8e297a29d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:19.438314 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8c913e-f41e-4a31-98f6-7f8e297a29d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:19.438590 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8c913e-f41e-4a31-98f6-7f8e297a29d7 admin admin] PATH_INFO: `/v3/users/13d635046ede46f8bb6e8e43ae83b577` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:19.439783 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8c913e-f41e-4a31-98f6-7f8e297a29d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13d635046ede46f8bb6e8e43ae83b577)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:19.441739 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8c913e-f41e-4a31-98f6-7f8e297a29d7 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:19.493975 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-8b8c913e-f41e-4a31-98f6-7f8e297a29d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13d635046ede46f8bb6e8e43ae83b577'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:19.507401 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-8b8c913e-f41e-4a31-98f6-7f8e297a29d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13d635046ede46f8bb6e8e43ae83b577'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:19.508852 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-8b8c913e-f41e-4a31-98f6-7f8e297a29d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13d635046ede46f8bb6e8e43ae83b577'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:19.516841 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-8b8c913e-f41e-4a31-98f6-7f8e297a29d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13d635046ede46f8bb6e8e43ae83b577'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:19.517546 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-8b8c913e-f41e-4a31-98f6-7f8e297a29d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13d635046ede46f8bb6e8e43ae83b577'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:19.533182 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1393/2810] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:52:19 2022] DELETE /identity/v3/users/13d635046ede46f8bb6e8e43ae83b577 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:52:19.537759 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac3c17-9947-4b39-bfc4-7db8bda490fc None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:19.538175 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac3c17-9947-4b39-bfc4-7db8bda490fc None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:19.538714 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac3c17-9947-4b39-bfc4-7db8bda490fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:19.558490 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-a0ac3c17-9947-4b39-bfc4-7db8bda490fc None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:52:19.559709 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-a0ac3c17-9947-4b39-bfc4-7db8bda490fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:19.606116 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:52:19.622600 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1418/2811] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:52:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:52:19.633207 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8c913e-f41e-4a31-98f6-7f8e297a29d7 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:19.634722 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-8b8c913e-f41e-4a31-98f6-7f8e297a29d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:19.693638 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8c913e-f41e-4a31-98f6-7f8e297a29d7 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:19.698054 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50c727e-5f85-42d1-8669-03ffe44cca34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:19.698701 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c727e-5f85-42d1-8669-03ffe44cca34 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:19.698987 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c727e-5f85-42d1-8669-03ffe44cca34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:19.699291 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c727e-5f85-42d1-8669-03ffe44cca34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:19.709154 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50c727e-5f85-42d1-8669-03ffe44cca34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:19.710697 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50c727e-5f85-42d1-8669-03ffe44cca34 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:19.720424 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1394/2812] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:52:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:52:20.003516 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ac3c17-9947-4b39-bfc4-7db8bda490fc None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:20.005317 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-a0ac3c17-9947-4b39-bfc4-7db8bda490fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:20.019235 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ac3c17-9947-4b39-bfc4-7db8bda490fc None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:20.024844 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c90f017-9dcd-41a0-ac9a-5744a0880e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:20.025817 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c90f017-9dcd-41a0-ac9a-5744a0880e56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:20.026021 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c90f017-9dcd-41a0-ac9a-5744a0880e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:20.026299 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c90f017-9dcd-41a0-ac9a-5744a0880e56 admin admin] PATH_INFO: `/v3/projects/95a26ac1eb1a4434b42be05c98931c82` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:20.027846 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c90f017-9dcd-41a0-ac9a-5744a0880e56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95a26ac1eb1a4434b42be05c98931c82)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:20.029787 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c90f017-9dcd-41a0-ac9a-5744a0880e56 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:20.079724 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-8c90f017-9dcd-41a0-ac9a-5744a0880e56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95a26ac1eb1a4434b42be05c98931c82', 'request_id': 'req-8c90f017-9dcd-41a0-ac9a-5744a0880e56'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:20.087471 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-8c90f017-9dcd-41a0-ac9a-5744a0880e56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95a26ac1eb1a4434b42be05c98931c82', 'request_id': 'req-8c90f017-9dcd-41a0-ac9a-5744a0880e56'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:20.093221 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-8c90f017-9dcd-41a0-ac9a-5744a0880e56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95a26ac1eb1a4434b42be05c98931c82', 'request_id': 'req-8c90f017-9dcd-41a0-ac9a-5744a0880e56'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:20.107031 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-8c90f017-9dcd-41a0-ac9a-5744a0880e56 admin admin] The token cache is being invalidate because project 95a26ac1eb1a4434b42be05c98931c82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:52:20.107317 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-8c90f017-9dcd-41a0-ac9a-5744a0880e56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:20.108143 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-8c90f017-9dcd-41a0-ac9a-5744a0880e56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:20.109742 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1419/2813] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:52:20 2022] DELETE /identity/v3/projects/95a26ac1eb1a4434b42be05c98931c82 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:52:20.113791 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50c727e-5f85-42d1-8669-03ffe44cca34 service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:20.125027 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50c727e-5f85-42d1-8669-03ffe44cca34 service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:20.141591 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:52:20.186614 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4d64c9-3d9f-4a1f-a758-cb17ab51c736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:20.187499 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d64c9-3d9f-4a1f-a758-cb17ab51c736 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:20.187780 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d64c9-3d9f-4a1f-a758-cb17ab51c736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:20.188052 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d64c9-3d9f-4a1f-a758-cb17ab51c736 admin admin] PATH_INFO: `/v3/users/63513152a0104794b2aa724f7c671092` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:20.189230 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4d64c9-3d9f-4a1f-a758-cb17ab51c736 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63513152a0104794b2aa724f7c671092)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:20.191174 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4d64c9-3d9f-4a1f-a758-cb17ab51c736 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:20.237033 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-1f4d64c9-3d9f-4a1f-a758-cb17ab51c736 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63513152a0104794b2aa724f7c671092'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:20.251156 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-1f4d64c9-3d9f-4a1f-a758-cb17ab51c736 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63513152a0104794b2aa724f7c671092'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:20.251889 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-1f4d64c9-3d9f-4a1f-a758-cb17ab51c736 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63513152a0104794b2aa724f7c671092'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:20.261063 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-1f4d64c9-3d9f-4a1f-a758-cb17ab51c736 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63513152a0104794b2aa724f7c671092'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:20.261885 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-1f4d64c9-3d9f-4a1f-a758-cb17ab51c736 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63513152a0104794b2aa724f7c671092'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:20.278068 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1395/2814] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:52:20 2022] DELETE /identity/v3/users/63513152a0104794b2aa724f7c671092 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:52:20.566499 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c90f017-9dcd-41a0-ac9a-5744a0880e56 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:20.568427 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-8c90f017-9dcd-41a0-ac9a-5744a0880e56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:20.586025 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c90f017-9dcd-41a0-ac9a-5744a0880e56 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:20.652279 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8892402b-ddc1-45dd-83d0-a5751213e5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:20.653164 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8892402b-ddc1-45dd-83d0-a5751213e5a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:20.653454 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8892402b-ddc1-45dd-83d0-a5751213e5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:20.653752 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8892402b-ddc1-45dd-83d0-a5751213e5a6 admin admin] PATH_INFO: `/v3/projects/6090bd704a8e46d89913ef118d6bdfcd` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:20.656696 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8892402b-ddc1-45dd-83d0-a5751213e5a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6090bd704a8e46d89913ef118d6bdfcd)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:20.658762 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8892402b-ddc1-45dd-83d0-a5751213e5a6 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:20.715086 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-8892402b-ddc1-45dd-83d0-a5751213e5a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6090bd704a8e46d89913ef118d6bdfcd', 'request_id': 'req-8892402b-ddc1-45dd-83d0-a5751213e5a6'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:20.722875 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-8892402b-ddc1-45dd-83d0-a5751213e5a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6090bd704a8e46d89913ef118d6bdfcd', 'request_id': 'req-8892402b-ddc1-45dd-83d0-a5751213e5a6'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:20.730144 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-8892402b-ddc1-45dd-83d0-a5751213e5a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6090bd704a8e46d89913ef118d6bdfcd', 'request_id': 'req-8892402b-ddc1-45dd-83d0-a5751213e5a6'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:20.744433 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-8892402b-ddc1-45dd-83d0-a5751213e5a6 admin admin] The token cache is being invalidate because project 6090bd704a8e46d89913ef118d6bdfcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:52:20.744988 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-8892402b-ddc1-45dd-83d0-a5751213e5a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:20.745733 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-8892402b-ddc1-45dd-83d0-a5751213e5a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:20.747365 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1420/2815] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:52:20 2022] DELETE /identity/v3/projects/6090bd704a8e46d89913ef118d6bdfcd => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:52:20.752803 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd6481c-effb-4dc6-9980-b869eac39f6b None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:20.753271 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd6481c-effb-4dc6-9980-b869eac39f6b None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:20.753381 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd6481c-effb-4dc6-9980-b869eac39f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:20.773833 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-2fd6481c-effb-4dc6-9980-b869eac39f6b None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:52:20.775062 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-2fd6481c-effb-4dc6-9980-b869eac39f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:20.834217 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1396/2816] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:52:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:52:20.842548 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f862562-6da9-4e67-a2ac-b7f7ceb72866 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:20.842844 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f862562-6da9-4e67-a2ac-b7f7ceb72866 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:20.843131 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f862562-6da9-4e67-a2ac-b7f7ceb72866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:20.850712 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:52:20.864564 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-6f862562-6da9-4e67-a2ac-b7f7ceb72866 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:52:20.866131 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-6f862562-6da9-4e67-a2ac-b7f7ceb72866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:20.874239 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1421/2817] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:52:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:52:20.878491 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd6481c-effb-4dc6-9980-b869eac39f6b None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:20.890045 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd6481c-effb-4dc6-9980-b869eac39f6b None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:20.894647 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87709ab4-c9bf-4578-b092-ffd36e13c6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:20.895311 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87709ab4-c9bf-4578-b092-ffd36e13c6f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:20.895599 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87709ab4-c9bf-4578-b092-ffd36e13c6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:20.895876 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87709ab4-c9bf-4578-b092-ffd36e13c6f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:20.896601 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87709ab4-c9bf-4578-b092-ffd36e13c6f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:52:20.896906 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87709ab4-c9bf-4578-b092-ffd36e13c6f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:20.899171 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87709ab4-c9bf-4578-b092-ffd36e13c6f1 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:20.911107 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1397/2818] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:52:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:52:20.916107 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f862562-6da9-4e67-a2ac-b7f7ceb72866 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:20.925548 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f862562-6da9-4e67-a2ac-b7f7ceb72866 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:20.930572 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab1e4f-fd3e-4666-a32b-ecfe99c4e4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:20.931225 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab1e4f-fd3e-4666-a32b-ecfe99c4e4b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:20.931511 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab1e4f-fd3e-4666-a32b-ecfe99c4e4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:20.931789 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab1e4f-fd3e-4666-a32b-ecfe99c4e4b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:20.932471 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ab1e4f-fd3e-4666-a32b-ecfe99c4e4b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:20.934415 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ab1e4f-fd3e-4666-a32b-ecfe99c4e4b8 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:20.962403 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1422/2819] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:52:20 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:52:20.966364 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87709ab4-c9bf-4578-b092-ffd36e13c6f1 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:20.976720 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87709ab4-c9bf-4578-b092-ffd36e13c6f1 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:21.001018 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:52:21.037428 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a5d07-7a90-44f8-97c0-fec5c4ad7f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:21.038304 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a5d07-7a90-44f8-97c0-fec5c4ad7f31 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:21.038590 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a5d07-7a90-44f8-97c0-fec5c4ad7f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:21.039081 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a5d07-7a90-44f8-97c0-fec5c4ad7f31 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:21.039778 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790a5d07-7a90-44f8-97c0-fec5c4ad7f31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:21.041720 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790a5d07-7a90-44f8-97c0-fec5c4ad7f31 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:21.062298 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1398/2820] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:52:20 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:52:21.066376 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab1e4f-fd3e-4666-a32b-ecfe99c4e4b8 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:21.078129 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab1e4f-fd3e-4666-a32b-ecfe99c4e4b8 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:21.084317 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df576b97-7509-4b54-b442-49a965dcd2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:21.085610 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df576b97-7509-4b54-b442-49a965dcd2de admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:21.085950 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df576b97-7509-4b54-b442-49a965dcd2de admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:21.086232 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df576b97-7509-4b54-b442-49a965dcd2de admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:21.086974 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df576b97-7509-4b54-b442-49a965dcd2de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:21.088619 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df576b97-7509-4b54-b442-49a965dcd2de admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:21.100319 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1423/2821] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:52:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:52:21.104849 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a5d07-7a90-44f8-97c0-fec5c4ad7f31 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:21.118061 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a5d07-7a90-44f8-97c0-fec5c4ad7f31 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:21.121969 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b02be4a-b0e1-487f-85f2-63f2f9a8400c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:21.122636 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b02be4a-b0e1-487f-85f2-63f2f9a8400c admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:21.122921 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b02be4a-b0e1-487f-85f2-63f2f9a8400c admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:21.123211 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b02be4a-b0e1-487f-85f2-63f2f9a8400c admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:21.123933 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b02be4a-b0e1-487f-85f2-63f2f9a8400c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:21.125544 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b02be4a-b0e1-487f-85f2-63f2f9a8400c admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:21.137585 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1399/2822] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:52:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:52:21.141651 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df576b97-7509-4b54-b442-49a965dcd2de admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:21.155087 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df576b97-7509-4b54-b442-49a965dcd2de admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:21.159898 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9ee64b-889f-4207-928f-c953dfd68140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:21.160931 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ee64b-889f-4207-928f-c953dfd68140 admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:21.161328 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ee64b-889f-4207-928f-c953dfd68140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:21.161580 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ee64b-889f-4207-928f-c953dfd68140 admin admin] PATH_INFO: `/v3/projects/e41f4aa82d634098a620b11ced421e5d/users/a275a05d40aa4ac3b85ca2db2b5bcf7a/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:21.177779 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9ee64b-889f-4207-928f-c953dfd68140 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e41f4aa82d634098a620b11ced421e5d, user_id=a275a05d40aa4ac3b85ca2db2b5bcf7a, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:21.181119 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9ee64b-889f-4207-928f-c953dfd68140 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:21.195489 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1424/2823] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:52:21 2022] PUT /identity/v3/projects/e41f4aa82d634098a620b11ced421e5d/users/a275a05d40aa4ac3b85ca2db2b5bcf7a/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:52:21.207261 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3350e418-30fa-4aaf-b556-a1477edfddb3 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:21.207554 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3350e418-30fa-4aaf-b556-a1477edfddb3 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:21.207867 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3350e418-30fa-4aaf-b556-a1477edfddb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:21.234672 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-3350e418-30fa-4aaf-b556-a1477edfddb3 None None] MFA Rules not processed for user `a275a05d40aa4ac3b85ca2db2b5bcf7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:52:21.235940 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-3350e418-30fa-4aaf-b556-a1477edfddb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:21.312159 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1400/2824] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:52:21 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:52:21.316893 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9ee64b-889f-4207-928f-c953dfd68140 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:21.383816 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9ee64b-889f-4207-928f-c953dfd68140 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:21.388473 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6836e83-eaa6-4b4c-90bc-053bf80790d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:21.389125 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6836e83-eaa6-4b4c-90bc-053bf80790d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:21.389410 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6836e83-eaa6-4b4c-90bc-053bf80790d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:21.389836 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6836e83-eaa6-4b4c-90bc-053bf80790d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:21.390526 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6836e83-eaa6-4b4c-90bc-053bf80790d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:21.392400 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6836e83-eaa6-4b4c-90bc-053bf80790d8 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:21.417860 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1425/2825] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:52:21 2022] POST /identity/v3/projects => generated 381 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:52:21.421825 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3350e418-30fa-4aaf-b556-a1477edfddb3 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:21.433772 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3350e418-30fa-4aaf-b556-a1477edfddb3 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:21.494971 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2efe703-80c4-4c87-b18b-704d4c3b6e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:21.497641 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2efe703-80c4-4c87-b18b-704d4c3b6e99 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:21.497962 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2efe703-80c4-4c87-b18b-704d4c3b6e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:21.498243 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2efe703-80c4-4c87-b18b-704d4c3b6e99 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:21.498948 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2efe703-80c4-4c87-b18b-704d4c3b6e99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:21.500884 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2efe703-80c4-4c87-b18b-704d4c3b6e99 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:21.522486 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1401/2826] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:52:21 2022] POST /identity/v3/users => generated 297 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:52:21.526661 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6836e83-eaa6-4b4c-90bc-053bf80790d8 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:21.537203 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6836e83-eaa6-4b4c-90bc-053bf80790d8 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:21.542453 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c613c3-c37d-4c8c-8b18-d2032b1a928d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:21.543110 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c613c3-c37d-4c8c-8b18-d2032b1a928d admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:21.543528 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c613c3-c37d-4c8c-8b18-d2032b1a928d admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:21.543799 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c613c3-c37d-4c8c-8b18-d2032b1a928d admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:21.544522 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c613c3-c37d-4c8c-8b18-d2032b1a928d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:21.546159 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c613c3-c37d-4c8c-8b18-d2032b1a928d admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:21.558420 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1426/2827] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:52:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:52:21.562909 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2efe703-80c4-4c87-b18b-704d4c3b6e99 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:21.574674 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2efe703-80c4-4c87-b18b-704d4c3b6e99 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:21.579812 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2e40d7-a782-49cb-899a-4b01ac4ab396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:21.580777 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e40d7-a782-49cb-899a-4b01ac4ab396 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:21.581061 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e40d7-a782-49cb-899a-4b01ac4ab396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:21.581336 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e40d7-a782-49cb-899a-4b01ac4ab396 admin admin] PATH_INFO: `/v3/projects/08973f890cef4a9d9cd70b521e4a10f6/users/2a86b99167234ac6ac8171183cfb77e0/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:21.598858 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2e40d7-a782-49cb-899a-4b01ac4ab396 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08973f890cef4a9d9cd70b521e4a10f6, user_id=2a86b99167234ac6ac8171183cfb77e0, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:21.602186 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2e40d7-a782-49cb-899a-4b01ac4ab396 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:21.612619 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:52:21.619474 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1402/2828] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:52:21 2022] PUT /identity/v3/projects/08973f890cef4a9d9cd70b521e4a10f6/users/2a86b99167234ac6ac8171183cfb77e0/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:52:21.623891 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c613c3-c37d-4c8c-8b18-d2032b1a928d admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:21.634991 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c613c3-c37d-4c8c-8b18-d2032b1a928d admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:21.693976 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95a266a-89ec-41bf-b5aa-8dfec8edea6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:21.694666 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95a266a-89ec-41bf-b5aa-8dfec8edea6f admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:21.694947 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95a266a-89ec-41bf-b5aa-8dfec8edea6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:21.695221 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95a266a-89ec-41bf-b5aa-8dfec8edea6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:21.695932 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95a266a-89ec-41bf-b5aa-8dfec8edea6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:21.697518 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95a266a-89ec-41bf-b5aa-8dfec8edea6f admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:21.707203 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1427/2829] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:52:21 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:52:21.711704 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2e40d7-a782-49cb-899a-4b01ac4ab396 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:21.722182 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2e40d7-a782-49cb-899a-4b01ac4ab396 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:21.727452 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c5dd21-8b0b-4f3d-91fd-30c507ce080c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:21.728333 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5dd21-8b0b-4f3d-91fd-30c507ce080c admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:21.728618 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5dd21-8b0b-4f3d-91fd-30c507ce080c admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:21.728897 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5dd21-8b0b-4f3d-91fd-30c507ce080c admin admin] PATH_INFO: `/v3/projects/08973f890cef4a9d9cd70b521e4a10f6/users/2a86b99167234ac6ac8171183cfb77e0/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:21.730901 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c5dd21-8b0b-4f3d-91fd-30c507ce080c admin admin] RBAC: Authorizing `identity:create_grant(project_id=08973f890cef4a9d9cd70b521e4a10f6, user_id=2a86b99167234ac6ac8171183cfb77e0, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:21.733528 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c5dd21-8b0b-4f3d-91fd-30c507ce080c admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:21.744575 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1403/2830] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:52:21 2022] PUT /identity/v3/projects/08973f890cef4a9d9cd70b521e4a10f6/users/2a86b99167234ac6ac8171183cfb77e0/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:52:21.755146 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d16bdf0-a98e-4424-a86c-c99453cc6f49 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:21.755465 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d16bdf0-a98e-4424-a86c-c99453cc6f49 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:21.755746 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d16bdf0-a98e-4424-a86c-c99453cc6f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:21.778859 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:52:21.791674 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-3d16bdf0-a98e-4424-a86c-c99453cc6f49 None None] MFA Rules not processed for user `2a86b99167234ac6ac8171183cfb77e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:52:21.793585 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-3d16bdf0-a98e-4424-a86c-c99453cc6f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:21.881417 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1428/2831] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:52:21 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:52:21.895865 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c5dd21-8b0b-4f3d-91fd-30c507ce080c admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:21.898947 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-36c5dd21-8b0b-4f3d-91fd-30c507ce080c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:21.980270 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c5dd21-8b0b-4f3d-91fd-30c507ce080c admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:21.985433 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2939633-2c91-4991-9748-a3d52b2f9978 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:21.986185 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2939633-2c91-4991-9748-a3d52b2f9978 service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:21.986651 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2939633-2c91-4991-9748-a3d52b2f9978 service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:21.986962 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2939633-2c91-4991-9748-a3d52b2f9978 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:21.999478 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2939633-2c91-4991-9748-a3d52b2f9978 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:22.001425 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2939633-2c91-4991-9748-a3d52b2f9978 service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:22.012522 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1404/2832] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:52:21 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:52:22.375756 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d16bdf0-a98e-4424-a86c-c99453cc6f49 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:22.386028 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d16bdf0-a98e-4424-a86c-c99453cc6f49 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:22.390668 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfacd9d-abfa-48b1-8445-3e6a2ec1b440 tempest-VolumeQuotaClassesTest-1038449655 tempest-VolumeQuotaClassesTest-1038449655-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a86b99167234ac6ac8171183cfb77e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08973f890cef4a9d9cd70b521e4a10f6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:22.391358 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfacd9d-abfa-48b1-8445-3e6a2ec1b440 tempest-VolumeQuotaClassesTest-1038449655 tempest-VolumeQuotaClassesTest-1038449655-project-admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:22.391644 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfacd9d-abfa-48b1-8445-3e6a2ec1b440 tempest-VolumeQuotaClassesTest-1038449655 tempest-VolumeQuotaClassesTest-1038449655-project-admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:22.391930 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfacd9d-abfa-48b1-8445-3e6a2ec1b440 tempest-VolumeQuotaClassesTest-1038449655 tempest-VolumeQuotaClassesTest-1038449655-project-admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:22.392688 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfacd9d-abfa-48b1-8445-3e6a2ec1b440 tempest-VolumeQuotaClassesTest-1038449655 tempest-VolumeQuotaClassesTest-1038449655-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:52:22.392994 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfacd9d-abfa-48b1-8445-3e6a2ec1b440 tempest-VolumeQuotaClassesTest-1038449655 tempest-VolumeQuotaClassesTest-1038449655-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:22.394630 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfacd9d-abfa-48b1-8445-3e6a2ec1b440 tempest-VolumeQuotaClassesTest-1038449655 tempest-VolumeQuotaClassesTest-1038449655-project-admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:22.409044 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1429/2833] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:52:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:52:22.413123 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2939633-2c91-4991-9748-a3d52b2f9978 service cinder] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:22.422672 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2939633-2c91-4991-9748-a3d52b2f9978 service cinder] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:22.427630 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37692b1-9490-42ec-add7-3987e8cb4bf7 tempest-VolumeQuotaClassesTest-1038449655 tempest-VolumeQuotaClassesTest-1038449655-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a86b99167234ac6ac8171183cfb77e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08973f890cef4a9d9cd70b521e4a10f6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:22.428303 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37692b1-9490-42ec-add7-3987e8cb4bf7 tempest-VolumeQuotaClassesTest-1038449655 tempest-VolumeQuotaClassesTest-1038449655-project-admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:22.428688 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37692b1-9490-42ec-add7-3987e8cb4bf7 tempest-VolumeQuotaClassesTest-1038449655 tempest-VolumeQuotaClassesTest-1038449655-project-admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:22.429593 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37692b1-9490-42ec-add7-3987e8cb4bf7 tempest-VolumeQuotaClassesTest-1038449655 tempest-VolumeQuotaClassesTest-1038449655-project-admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:22.430333 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37692b1-9490-42ec-add7-3987e8cb4bf7 tempest-VolumeQuotaClassesTest-1038449655 tempest-VolumeQuotaClassesTest-1038449655-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:22.432256 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37692b1-9490-42ec-add7-3987e8cb4bf7 tempest-VolumeQuotaClassesTest-1038449655 tempest-VolumeQuotaClassesTest-1038449655-project-admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:22.461764 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1405/2834] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:52:22 2022] POST /identity/v3/projects => generated 354 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:52:22.466602 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfacd9d-abfa-48b1-8445-3e6a2ec1b440 tempest-VolumeQuotaClassesTest-1038449655 tempest-VolumeQuotaClassesTest-1038449655-project-admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:22.478249 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfacd9d-abfa-48b1-8445-3e6a2ec1b440 tempest-VolumeQuotaClassesTest-1038449655 tempest-VolumeQuotaClassesTest-1038449655-project-admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:22.543512 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04a667-3080-4f88-97b3-47beb1166233 tempest-VolumeQuotaClassesTest-1038449655 tempest-VolumeQuotaClassesTest-1038449655-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a86b99167234ac6ac8171183cfb77e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08973f890cef4a9d9cd70b521e4a10f6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:22.544187 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04a667-3080-4f88-97b3-47beb1166233 tempest-VolumeQuotaClassesTest-1038449655 tempest-VolumeQuotaClassesTest-1038449655-project-admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:22.544480 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04a667-3080-4f88-97b3-47beb1166233 tempest-VolumeQuotaClassesTest-1038449655 tempest-VolumeQuotaClassesTest-1038449655-project-admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:22.544761 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04a667-3080-4f88-97b3-47beb1166233 tempest-VolumeQuotaClassesTest-1038449655 tempest-VolumeQuotaClassesTest-1038449655-project-admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:22.545487 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d04a667-3080-4f88-97b3-47beb1166233 tempest-VolumeQuotaClassesTest-1038449655 tempest-VolumeQuotaClassesTest-1038449655-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:52:22.545830 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d04a667-3080-4f88-97b3-47beb1166233 tempest-VolumeQuotaClassesTest-1038449655 tempest-VolumeQuotaClassesTest-1038449655-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:22.547429 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d04a667-3080-4f88-97b3-47beb1166233 tempest-VolumeQuotaClassesTest-1038449655 tempest-VolumeQuotaClassesTest-1038449655-project-admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:22.560251 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1430/2835] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:52:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:52:22.600148 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37692b1-9490-42ec-add7-3987e8cb4bf7 tempest-VolumeQuotaClassesTest-1038449655 tempest-VolumeQuotaClassesTest-1038449655-project-admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:22.613231 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37692b1-9490-42ec-add7-3987e8cb4bf7 tempest-VolumeQuotaClassesTest-1038449655 tempest-VolumeQuotaClassesTest-1038449655-project-admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:22.619291 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1839558f-7a64-46ae-a365-876d0ca0cd5d tempest-VolumeQuotaClassesTest-1038449655 tempest-VolumeQuotaClassesTest-1038449655-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a86b99167234ac6ac8171183cfb77e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08973f890cef4a9d9cd70b521e4a10f6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:22.620193 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839558f-7a64-46ae-a365-876d0ca0cd5d tempest-VolumeQuotaClassesTest-1038449655 tempest-VolumeQuotaClassesTest-1038449655-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:22.620483 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839558f-7a64-46ae-a365-876d0ca0cd5d tempest-VolumeQuotaClassesTest-1038449655 tempest-VolumeQuotaClassesTest-1038449655-project-admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:22.620761 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839558f-7a64-46ae-a365-876d0ca0cd5d tempest-VolumeQuotaClassesTest-1038449655 tempest-VolumeQuotaClassesTest-1038449655-project-admin] PATH_INFO: `/v3/projects/bead29c54d144186b9f7c9b0c3f2aaa5` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:22.622131 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1839558f-7a64-46ae-a365-876d0ca0cd5d tempest-VolumeQuotaClassesTest-1038449655 tempest-VolumeQuotaClassesTest-1038449655-project-admin] RBAC: Authorizing `identity:delete_project(project_id=bead29c54d144186b9f7c9b0c3f2aaa5)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:22.624110 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1839558f-7a64-46ae-a365-876d0ca0cd5d tempest-VolumeQuotaClassesTest-1038449655 tempest-VolumeQuotaClassesTest-1038449655-project-admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:22.680358 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-1839558f-7a64-46ae-a365-876d0ca0cd5d tempest-VolumeQuotaClassesTest-1038449655 tempest-VolumeQuotaClassesTest-1038449655-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bead29c54d144186b9f7c9b0c3f2aaa5', 'request_id': 'req-1839558f-7a64-46ae-a365-876d0ca0cd5d'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:22.687790 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-1839558f-7a64-46ae-a365-876d0ca0cd5d tempest-VolumeQuotaClassesTest-1038449655 tempest-VolumeQuotaClassesTest-1038449655-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bead29c54d144186b9f7c9b0c3f2aaa5', 'request_id': 'req-1839558f-7a64-46ae-a365-876d0ca0cd5d'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:22.693978 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-1839558f-7a64-46ae-a365-876d0ca0cd5d tempest-VolumeQuotaClassesTest-1038449655 tempest-VolumeQuotaClassesTest-1038449655-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bead29c54d144186b9f7c9b0c3f2aaa5', 'request_id': 'req-1839558f-7a64-46ae-a365-876d0ca0cd5d'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:22.709816 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-1839558f-7a64-46ae-a365-876d0ca0cd5d tempest-VolumeQuotaClassesTest-1038449655 tempest-VolumeQuotaClassesTest-1038449655-project-admin] The token cache is being invalidate because project bead29c54d144186b9f7c9b0c3f2aaa5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:52:22.710267 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-1839558f-7a64-46ae-a365-876d0ca0cd5d tempest-VolumeQuotaClassesTest-1038449655 tempest-VolumeQuotaClassesTest-1038449655-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:22.711265 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-1839558f-7a64-46ae-a365-876d0ca0cd5d tempest-VolumeQuotaClassesTest-1038449655 tempest-VolumeQuotaClassesTest-1038449655-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:22.713083 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1406/2836] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:52:22 2022] DELETE /identity/v3/projects/bead29c54d144186b9f7c9b0c3f2aaa5 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:52:22.951753 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04a667-3080-4f88-97b3-47beb1166233 tempest-VolumeQuotaClassesTest-1038449655 tempest-VolumeQuotaClassesTest-1038449655-project-admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:22.953553 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-2d04a667-3080-4f88-97b3-47beb1166233 tempest-VolumeQuotaClassesTest-1038449655 tempest-VolumeQuotaClassesTest-1038449655-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:23.026772 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04a667-3080-4f88-97b3-47beb1166233 tempest-VolumeQuotaClassesTest-1038449655 tempest-VolumeQuotaClassesTest-1038449655-project-admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:23.052991 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:52:23.115739 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a346cab0-da78-4c34-b4df-dd3d9d0e3260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:23.116896 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346cab0-da78-4c34-b4df-dd3d9d0e3260 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:23.116896 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346cab0-da78-4c34-b4df-dd3d9d0e3260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:23.116896 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346cab0-da78-4c34-b4df-dd3d9d0e3260 admin admin] PATH_INFO: `/v3/users/a275a05d40aa4ac3b85ca2db2b5bcf7a` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:23.118306 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a346cab0-da78-4c34-b4df-dd3d9d0e3260 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a275a05d40aa4ac3b85ca2db2b5bcf7a)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:23.120313 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a346cab0-da78-4c34-b4df-dd3d9d0e3260 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:23.175274 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a346cab0-da78-4c34-b4df-dd3d9d0e3260 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a275a05d40aa4ac3b85ca2db2b5bcf7a'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:23.190132 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a346cab0-da78-4c34-b4df-dd3d9d0e3260 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a275a05d40aa4ac3b85ca2db2b5bcf7a'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:23.198272 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a346cab0-da78-4c34-b4df-dd3d9d0e3260 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a275a05d40aa4ac3b85ca2db2b5bcf7a'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:23.199007 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a346cab0-da78-4c34-b4df-dd3d9d0e3260 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a275a05d40aa4ac3b85ca2db2b5bcf7a'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:23.199647 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-a346cab0-da78-4c34-b4df-dd3d9d0e3260 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a275a05d40aa4ac3b85ca2db2b5bcf7a'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:23.213458 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1431/2837] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:52:22 2022] DELETE /identity/v3/users/a275a05d40aa4ac3b85ca2db2b5bcf7a => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:52:23.218160 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf3f8c-015c-4656-9b2b-89389171437d None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:23.218645 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf3f8c-015c-4656-9b2b-89389171437d None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:23.218777 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf3f8c-015c-4656-9b2b-89389171437d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:23.237249 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-3caf3f8c-015c-4656-9b2b-89389171437d None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:52:23.238513 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-3caf3f8c-015c-4656-9b2b-89389171437d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:23.298435 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:52:23.314947 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1407/2838] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:52:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:52:23.325107 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a346cab0-da78-4c34-b4df-dd3d9d0e3260 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:23.326575 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-a346cab0-da78-4c34-b4df-dd3d9d0e3260 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:23.390565 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a346cab0-da78-4c34-b4df-dd3d9d0e3260 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:23.394720 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c9e7cb-c840-4c2d-9468-b74fe5d80cf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:23.395413 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9e7cb-c840-4c2d-9468-b74fe5d80cf6 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:23.395724 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9e7cb-c840-4c2d-9468-b74fe5d80cf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:23.396000 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9e7cb-c840-4c2d-9468-b74fe5d80cf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:23.408597 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c9e7cb-c840-4c2d-9468-b74fe5d80cf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:23.410829 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c9e7cb-c840-4c2d-9468-b74fe5d80cf6 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:23.425258 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1432/2839] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:52:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:52:23.727664 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caf3f8c-015c-4656-9b2b-89389171437d None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:23.729239 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-3caf3f8c-015c-4656-9b2b-89389171437d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:23.743783 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caf3f8c-015c-4656-9b2b-89389171437d None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:23.749909 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9ea63e-7008-44ac-9362-9a8a27593898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:23.750805 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9ea63e-7008-44ac-9362-9a8a27593898 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:23.751092 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9ea63e-7008-44ac-9362-9a8a27593898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:23.751424 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9ea63e-7008-44ac-9362-9a8a27593898 admin admin] PATH_INFO: `/v3/projects/e41f4aa82d634098a620b11ced421e5d` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:23.755972 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9ea63e-7008-44ac-9362-9a8a27593898 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e41f4aa82d634098a620b11ced421e5d)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:23.755972 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9ea63e-7008-44ac-9362-9a8a27593898 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:23.821150 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-0a9ea63e-7008-44ac-9362-9a8a27593898 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e41f4aa82d634098a620b11ced421e5d', 'request_id': 'req-0a9ea63e-7008-44ac-9362-9a8a27593898'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:23.827568 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-0a9ea63e-7008-44ac-9362-9a8a27593898 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e41f4aa82d634098a620b11ced421e5d', 'request_id': 'req-0a9ea63e-7008-44ac-9362-9a8a27593898'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:23.833534 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-0a9ea63e-7008-44ac-9362-9a8a27593898 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e41f4aa82d634098a620b11ced421e5d', 'request_id': 'req-0a9ea63e-7008-44ac-9362-9a8a27593898'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:23.849104 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-0a9ea63e-7008-44ac-9362-9a8a27593898 admin admin] The token cache is being invalidate because project e41f4aa82d634098a620b11ced421e5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:52:23.849392 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-0a9ea63e-7008-44ac-9362-9a8a27593898 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:23.850048 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-0a9ea63e-7008-44ac-9362-9a8a27593898 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:23.852036 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1408/2840] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:52:23 2022] DELETE /identity/v3/projects/e41f4aa82d634098a620b11ced421e5d => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:52:23.856182 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c9e7cb-c840-4c2d-9468-b74fe5d80cf6 service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:23.866995 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c9e7cb-c840-4c2d-9468-b74fe5d80cf6 service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:23.912489 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:52:23.924609 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9ea63e-7008-44ac-9362-9a8a27593898 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:23.926514 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-0a9ea63e-7008-44ac-9362-9a8a27593898 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:23.932121 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a61718-38b9-4c96-b77d-a43f4ea2d028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:23.933893 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a61718-38b9-4c96-b77d-a43f4ea2d028 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:23.934194 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a61718-38b9-4c96-b77d-a43f4ea2d028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:23.934475 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a61718-38b9-4c96-b77d-a43f4ea2d028 admin admin] PATH_INFO: `/v3/users/2a86b99167234ac6ac8171183cfb77e0` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:23.936020 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a61718-38b9-4c96-b77d-a43f4ea2d028 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a86b99167234ac6ac8171183cfb77e0)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:23.938277 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a61718-38b9-4c96-b77d-a43f4ea2d028 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:24.000757 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c8a61718-38b9-4c96-b77d-a43f4ea2d028 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a86b99167234ac6ac8171183cfb77e0'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:24.014767 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c8a61718-38b9-4c96-b77d-a43f4ea2d028 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a86b99167234ac6ac8171183cfb77e0'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:24.018782 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9ea63e-7008-44ac-9362-9a8a27593898 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:24.024075 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c8a61718-38b9-4c96-b77d-a43f4ea2d028 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a86b99167234ac6ac8171183cfb77e0'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:24.024953 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c8a61718-38b9-4c96-b77d-a43f4ea2d028 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a86b99167234ac6ac8171183cfb77e0'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:24.025764 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-c8a61718-38b9-4c96-b77d-a43f4ea2d028 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a86b99167234ac6ac8171183cfb77e0'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:24.027302 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd0545c-95e3-4271-9c2b-0880a22a9849 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74bb1bc6821b4500b26f160ccae5b7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:24.028264 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0545c-95e3-4271-9c2b-0880a22a9849 service nova] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:24.028574 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0545c-95e3-4271-9c2b-0880a22a9849 service nova] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:24.028856 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0545c-95e3-4271-9c2b-0880a22a9849 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:24.030957 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-4dd0545c-95e3-4271-9c2b-0880a22a9849 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:24.039965 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1433/2841] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:52:23 2022] DELETE /identity/v3/users/2a86b99167234ac6ac8171183cfb77e0 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:52:24.045142 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd0545c-95e3-4271-9c2b-0880a22a9849 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:24.046773 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd0545c-95e3-4271-9c2b-0880a22a9849 service nova] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:24.058394 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1409/2842] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:52:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:52:24.399083 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a61718-38b9-4c96-b77d-a43f4ea2d028 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:24.401074 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-c8a61718-38b9-4c96-b77d-a43f4ea2d028 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:24.488962 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a61718-38b9-4c96-b77d-a43f4ea2d028 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:24.494163 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a675eaf-b70e-474c-9fcb-300ad803059c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:24.495022 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a675eaf-b70e-474c-9fcb-300ad803059c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:24.495297 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a675eaf-b70e-474c-9fcb-300ad803059c admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:24.495568 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a675eaf-b70e-474c-9fcb-300ad803059c admin admin] PATH_INFO: `/v3/projects/08973f890cef4a9d9cd70b521e4a10f6` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:24.496814 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a675eaf-b70e-474c-9fcb-300ad803059c admin admin] RBAC: Authorizing `identity:delete_project(project_id=08973f890cef4a9d9cd70b521e4a10f6)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:24.498696 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a675eaf-b70e-474c-9fcb-300ad803059c admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:24.548956 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7a675eaf-b70e-474c-9fcb-300ad803059c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08973f890cef4a9d9cd70b521e4a10f6', 'request_id': 'req-7a675eaf-b70e-474c-9fcb-300ad803059c'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:24.556120 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7a675eaf-b70e-474c-9fcb-300ad803059c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08973f890cef4a9d9cd70b521e4a10f6', 'request_id': 'req-7a675eaf-b70e-474c-9fcb-300ad803059c'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:24.562012 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7a675eaf-b70e-474c-9fcb-300ad803059c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08973f890cef4a9d9cd70b521e4a10f6', 'request_id': 'req-7a675eaf-b70e-474c-9fcb-300ad803059c'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:24.565646 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:52:24.576965 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7a675eaf-b70e-474c-9fcb-300ad803059c admin admin] The token cache is being invalidate because project 08973f890cef4a9d9cd70b521e4a10f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:52:24.577256 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7a675eaf-b70e-474c-9fcb-300ad803059c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:24.577911 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-7a675eaf-b70e-474c-9fcb-300ad803059c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:24.579471 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1434/2843] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:52:24 2022] DELETE /identity/v3/projects/08973f890cef4a9d9cd70b521e4a10f6 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:52:24.584440 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb4cd3-2ae8-4282-9c32-ee4c6d48ae72 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:24.585476 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb4cd3-2ae8-4282-9c32-ee4c6d48ae72 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:24.585476 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb4cd3-2ae8-4282-9c32-ee4c6d48ae72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:24.606518 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-f5cb4cd3-2ae8-4282-9c32-ee4c6d48ae72 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:52:24.607760 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-f5cb4cd3-2ae8-4282-9c32-ee4c6d48ae72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:24.664811 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:52:24.675386 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1410/2844] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:52:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:52:24.683421 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec06540b-9882-4dba-8ac8-069fb3c730dc None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:24.684503 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec06540b-9882-4dba-8ac8-069fb3c730dc None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:24.684503 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec06540b-9882-4dba-8ac8-069fb3c730dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:24.704416 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-ec06540b-9882-4dba-8ac8-069fb3c730dc None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:52:24.705655 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-ec06540b-9882-4dba-8ac8-069fb3c730dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:24.714622 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1435/2845] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:52:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:52:24.719018 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cb4cd3-2ae8-4282-9c32-ee4c6d48ae72 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:24.729920 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cb4cd3-2ae8-4282-9c32-ee4c6d48ae72 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:24.734962 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b7245f-1e81-40e3-bfe9-acf6ef6cd836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:24.735663 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b7245f-1e81-40e3-bfe9-acf6ef6cd836 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:24.735946 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b7245f-1e81-40e3-bfe9-acf6ef6cd836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:24.736226 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b7245f-1e81-40e3-bfe9-acf6ef6cd836 admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:24.736956 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b7245f-1e81-40e3-bfe9-acf6ef6cd836 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:52:24.737305 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b7245f-1e81-40e3-bfe9-acf6ef6cd836 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:24.739070 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b7245f-1e81-40e3-bfe9-acf6ef6cd836 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:24.752442 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1411/2846] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:52:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:52:24.757028 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec06540b-9882-4dba-8ac8-069fb3c730dc None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:24.766418 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec06540b-9882-4dba-8ac8-069fb3c730dc None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:24.771156 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dfd785-bca9-4594-96f3-b95a5be41ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:24.771800 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dfd785-bca9-4594-96f3-b95a5be41ef4 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:24.772632 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dfd785-bca9-4594-96f3-b95a5be41ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:24.772914 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dfd785-bca9-4594-96f3-b95a5be41ef4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:24.773751 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dfd785-bca9-4594-96f3-b95a5be41ef4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:24.775646 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dfd785-bca9-4594-96f3-b95a5be41ef4 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:24.803636 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1436/2847] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:52:24 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:52:24.807102 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b7245f-1e81-40e3-bfe9-acf6ef6cd836 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:24.817412 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b7245f-1e81-40e3-bfe9-acf6ef6cd836 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:24.879795 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9372e125-bd71-4ef9-8914-af2dde429366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:24.880477 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372e125-bd71-4ef9-8914-af2dde429366 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:24.880764 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372e125-bd71-4ef9-8914-af2dde429366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:24.881038 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372e125-bd71-4ef9-8914-af2dde429366 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:24.883034 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9372e125-bd71-4ef9-8914-af2dde429366 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:24.884916 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9372e125-bd71-4ef9-8914-af2dde429366 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:24.906632 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1412/2848] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:52:24 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:52:24.910876 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dfd785-bca9-4594-96f3-b95a5be41ef4 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:24.921589 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dfd785-bca9-4594-96f3-b95a5be41ef4 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:24.926470 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3afe923-011d-48d0-aa11-1d0bb4deade0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:24.927143 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3afe923-011d-48d0-aa11-1d0bb4deade0 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:24.927431 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3afe923-011d-48d0-aa11-1d0bb4deade0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:24.927713 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3afe923-011d-48d0-aa11-1d0bb4deade0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:24.928509 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3afe923-011d-48d0-aa11-1d0bb4deade0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:24.930251 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3afe923-011d-48d0-aa11-1d0bb4deade0 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:24.941165 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1437/2849] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:52:24.945917 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9372e125-bd71-4ef9-8914-af2dde429366 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:24.956287 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9372e125-bd71-4ef9-8914-af2dde429366 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:24.961604 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb82db14-7e18-470b-a61c-7978a75f10ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:24.962297 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb82db14-7e18-470b-a61c-7978a75f10ee admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:24.962583 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb82db14-7e18-470b-a61c-7978a75f10ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:24.962860 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb82db14-7e18-470b-a61c-7978a75f10ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:24.963588 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb82db14-7e18-470b-a61c-7978a75f10ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:24.965193 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb82db14-7e18-470b-a61c-7978a75f10ee admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:24.977957 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1413/2850] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:52:24.982056 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3afe923-011d-48d0-aa11-1d0bb4deade0 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:24.993334 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3afe923-011d-48d0-aa11-1d0bb4deade0 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:24.998337 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df50794-3e5c-4d1f-bb21-c8d79bfbe74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:24.999258 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df50794-3e5c-4d1f-bb21-c8d79bfbe74a admin admin] REQUEST_METHOD: `PUT` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:24.999545 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df50794-3e5c-4d1f-bb21-c8d79bfbe74a admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:24.999823 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df50794-3e5c-4d1f-bb21-c8d79bfbe74a admin admin] PATH_INFO: `/v3/projects/a9bbff1a84a34de580f0062eb0e41793/users/07ed6264d90b435da91586a6b2b2056a/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:25.015551 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df50794-3e5c-4d1f-bb21-c8d79bfbe74a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9bbff1a84a34de580f0062eb0e41793, user_id=07ed6264d90b435da91586a6b2b2056a, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:25.018247 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df50794-3e5c-4d1f-bb21-c8d79bfbe74a admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:25.033560 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1438/2851] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:52:24 2022] PUT /identity/v3/projects/a9bbff1a84a34de580f0062eb0e41793/users/07ed6264d90b435da91586a6b2b2056a/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:52:25.102164 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c3301-0617-4a55-be5d-8162351dbb24 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:25.102454 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c3301-0617-4a55-be5d-8162351dbb24 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:25.102734 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c3301-0617-4a55-be5d-8162351dbb24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:25.134541 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-b34c3301-0617-4a55-be5d-8162351dbb24 None None] MFA Rules not processed for user `07ed6264d90b435da91586a6b2b2056a`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:52:25.135825 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-b34c3301-0617-4a55-be5d-8162351dbb24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:25.205999 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1414/2852] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:52:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:52:25.211452 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df50794-3e5c-4d1f-bb21-c8d79bfbe74a admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:25.275340 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df50794-3e5c-4d1f-bb21-c8d79bfbe74a admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:25.280090 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58964bf4-354b-430b-95cd-2dd6a14c74d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:25.280751 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58964bf4-354b-430b-95cd-2dd6a14c74d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:25.281041 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58964bf4-354b-430b-95cd-2dd6a14c74d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:25.281323 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58964bf4-354b-430b-95cd-2dd6a14c74d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:25.282048 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58964bf4-354b-430b-95cd-2dd6a14c74d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:25.283948 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58964bf4-354b-430b-95cd-2dd6a14c74d9 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:25.310319 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1439/2853] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:52:25 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:52:25.314450 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c3301-0617-4a55-be5d-8162351dbb24 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:25.325975 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c3301-0617-4a55-be5d-8162351dbb24 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:25.389002 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:52:25.397123 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a3884-8731-4834-a887-07de814cfe02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:25.398157 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a3884-8731-4834-a887-07de814cfe02 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:25.398507 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a3884-8731-4834-a887-07de814cfe02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:25.398923 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a3884-8731-4834-a887-07de814cfe02 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:25.399890 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328a3884-8731-4834-a887-07de814cfe02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:25.401872 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328a3884-8731-4834-a887-07de814cfe02 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:25.424129 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1415/2854] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:52:25 2022] POST /identity/v3/users => generated 299 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:52:25.428107 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58964bf4-354b-430b-95cd-2dd6a14c74d9 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:25.439476 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58964bf4-354b-430b-95cd-2dd6a14c74d9 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:25.444888 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc091c-35c7-42e3-b424-1afab94c372b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:25.445618 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc091c-35c7-42e3-b424-1afab94c372b admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:25.445937 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc091c-35c7-42e3-b424-1afab94c372b admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:25.446286 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc091c-35c7-42e3-b424-1afab94c372b admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:25.446958 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bc091c-35c7-42e3-b424-1afab94c372b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:25.448563 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bc091c-35c7-42e3-b424-1afab94c372b admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:25.462894 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1440/2855] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:52:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:52:25.464057 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a3884-8731-4834-a887-07de814cfe02 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:25.474610 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a3884-8731-4834-a887-07de814cfe02 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:25.479944 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aa64b8-8b01-41d1-823f-39e94aa6cbb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:25.480854 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa64b8-8b01-41d1-823f-39e94aa6cbb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:25.481134 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa64b8-8b01-41d1-823f-39e94aa6cbb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:25.481417 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa64b8-8b01-41d1-823f-39e94aa6cbb7 admin admin] PATH_INFO: `/v3/projects/414897f0cb01474583918890223b7c9a/users/777b53b121dd4410b9d39654eb07b53b/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:25.509547 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aa64b8-8b01-41d1-823f-39e94aa6cbb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=414897f0cb01474583918890223b7c9a, user_id=777b53b121dd4410b9d39654eb07b53b, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:25.513523 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aa64b8-8b01-41d1-823f-39e94aa6cbb7 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:25.535014 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1416/2856] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:52:25 2022] PUT /identity/v3/projects/414897f0cb01474583918890223b7c9a/users/777b53b121dd4410b9d39654eb07b53b/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:52:25.538856 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc091c-35c7-42e3-b424-1afab94c372b admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:25.549733 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc091c-35c7-42e3-b424-1afab94c372b admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:25.616770 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dd2929-2648-468e-adb7-2b258f8ff7e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:25.617454 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd2929-2648-468e-adb7-2b258f8ff7e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:25.617776 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd2929-2648-468e-adb7-2b258f8ff7e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:25.618067 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd2929-2648-468e-adb7-2b258f8ff7e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:25.618810 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dd2929-2648-468e-adb7-2b258f8ff7e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:25.620417 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dd2929-2648-468e-adb7-2b258f8ff7e8 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:25.631138 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1441/2857] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:52:25 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:52:25.636127 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aa64b8-8b01-41d1-823f-39e94aa6cbb7 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:25.646955 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aa64b8-8b01-41d1-823f-39e94aa6cbb7 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:25.652342 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ebf253-2fdd-4db9-b3d8-96edcb704e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:25.653236 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ebf253-2fdd-4db9-b3d8-96edcb704e34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:25.653659 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ebf253-2fdd-4db9-b3d8-96edcb704e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:25.653981 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ebf253-2fdd-4db9-b3d8-96edcb704e34 admin admin] PATH_INFO: `/v3/projects/414897f0cb01474583918890223b7c9a/users/777b53b121dd4410b9d39654eb07b53b/roles/acfe707eee6944ad94d1915fc9ee7ea3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:25.656005 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ebf253-2fdd-4db9-b3d8-96edcb704e34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=414897f0cb01474583918890223b7c9a, user_id=777b53b121dd4410b9d39654eb07b53b, role_id=acfe707eee6944ad94d1915fc9ee7ea3)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:25.658708 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ebf253-2fdd-4db9-b3d8-96edcb704e34 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:25.670079 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1417/2858] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:52:25 2022] PUT /identity/v3/projects/414897f0cb01474583918890223b7c9a/users/777b53b121dd4410b9d39654eb07b53b/roles/acfe707eee6944ad94d1915fc9ee7ea3 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:52:25.680707 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633138e7-d468-49cc-8467-7e02789117d1 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:25.680995 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633138e7-d468-49cc-8467-7e02789117d1 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:25.681282 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633138e7-d468-49cc-8467-7e02789117d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:25.687519 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:52:25.712504 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-633138e7-d468-49cc-8467-7e02789117d1 None None] MFA Rules not processed for user `777b53b121dd4410b9d39654eb07b53b`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:52:25.713760 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-633138e7-d468-49cc-8467-7e02789117d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:25.787508 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ebf253-2fdd-4db9-b3d8-96edcb704e34 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:25.789319 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-78ebf253-2fdd-4db9-b3d8-96edcb704e34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:25.797440 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1442/2859] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:52:25 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:52:25.808259 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633138e7-d468-49cc-8467-7e02789117d1 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:25.809663 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-633138e7-d468-49cc-8467-7e02789117d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:25.857003 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ebf253-2fdd-4db9-b3d8-96edcb704e34 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:25.860834 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4c9bf1-28ab-4c76-9689-2c2b324f812a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:25.861472 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c9bf1-28ab-4c76-9689-2c2b324f812a service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:25.861787 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c9bf1-28ab-4c76-9689-2c2b324f812a service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:25.862076 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c9bf1-28ab-4c76-9689-2c2b324f812a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:25.863878 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-1f4c9bf1-28ab-4c76-9689-2c2b324f812a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:25.871766 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633138e7-d468-49cc-8467-7e02789117d1 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:25.875556 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049ca298-8eb3-4144-bde6-ce709980ca2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dcdfb5ca1541f9b4967a1f8ddbb34b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:25.876215 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049ca298-8eb3-4144-bde6-ce709980ca2b service cinder] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:25.876503 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049ca298-8eb3-4144-bde6-ce709980ca2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:25.876783 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049ca298-8eb3-4144-bde6-ce709980ca2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:25.887991 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049ca298-8eb3-4144-bde6-ce709980ca2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:25.889489 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049ca298-8eb3-4144-bde6-ce709980ca2b service cinder] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:25.901581 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1443/2860] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:52:25 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:52:25.934442 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4c9bf1-28ab-4c76-9689-2c2b324f812a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:25.935898 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4c9bf1-28ab-4c76-9689-2c2b324f812a service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:25.947473 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1418/2861] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:52:25 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:52:26.533387 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049ca298-8eb3-4144-bde6-ce709980ca2b service cinder] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:26.601156 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049ca298-8eb3-4144-bde6-ce709980ca2b service cinder] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:26.605901 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af5a767-5730-4b2c-8dca-929283144681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:26.606788 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5a767-5730-4b2c-8dca-929283144681 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:26.607076 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5a767-5730-4b2c-8dca-929283144681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:26.607354 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5a767-5730-4b2c-8dca-929283144681 admin admin] PATH_INFO: `/v3/users/07ed6264d90b435da91586a6b2b2056a` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:26.608602 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af5a767-5730-4b2c-8dca-929283144681 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07ed6264d90b435da91586a6b2b2056a)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:26.610611 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af5a767-5730-4b2c-8dca-929283144681 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:26.651998 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:52:26.667817 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-9af5a767-5730-4b2c-8dca-929283144681 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07ed6264d90b435da91586a6b2b2056a'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:26.682482 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-9af5a767-5730-4b2c-8dca-929283144681 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07ed6264d90b435da91586a6b2b2056a'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:26.691097 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-9af5a767-5730-4b2c-8dca-929283144681 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07ed6264d90b435da91586a6b2b2056a'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:26.691720 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-9af5a767-5730-4b2c-8dca-929283144681 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07ed6264d90b435da91586a6b2b2056a'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:26.692319 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-9af5a767-5730-4b2c-8dca-929283144681 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07ed6264d90b435da91586a6b2b2056a'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:26.705950 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1444/2862] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:52:26 2022] DELETE /identity/v3/users/07ed6264d90b435da91586a6b2b2056a => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:52:26.710583 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db674842-11ee-4361-9ff4-eb9c6b5a5e84 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:26.711253 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db674842-11ee-4361-9ff4-eb9c6b5a5e84 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:26.711253 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db674842-11ee-4361-9ff4-eb9c6b5a5e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:26.729472 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-db674842-11ee-4361-9ff4-eb9c6b5a5e84 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:52:26.730760 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-db674842-11ee-4361-9ff4-eb9c6b5a5e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:26.788937 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:52:26.801998 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1419/2863] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:52:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:52:26.813088 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af5a767-5730-4b2c-8dca-929283144681 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:26.816169 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-9af5a767-5730-4b2c-8dca-929283144681 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:26.890400 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af5a767-5730-4b2c-8dca-929283144681 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:26.894878 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73622ea6-8c49-4c92-a5c2-1d2a39821cdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:26.895696 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73622ea6-8c49-4c92-a5c2-1d2a39821cdb service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:26.896277 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73622ea6-8c49-4c92-a5c2-1d2a39821cdb service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:26.896676 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73622ea6-8c49-4c92-a5c2-1d2a39821cdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:26.909644 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73622ea6-8c49-4c92-a5c2-1d2a39821cdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:26.911308 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73622ea6-8c49-4c92-a5c2-1d2a39821cdb service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:26.923256 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1445/2864] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:52:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:52:27.233895 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db674842-11ee-4361-9ff4-eb9c6b5a5e84 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:27.235571 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-db674842-11ee-4361-9ff4-eb9c6b5a5e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:27.250716 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db674842-11ee-4361-9ff4-eb9c6b5a5e84 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:27.257059 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffccf81-9153-47f2-832d-d7a5ce07a629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:27.257969 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffccf81-9153-47f2-832d-d7a5ce07a629 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:27.258274 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffccf81-9153-47f2-832d-d7a5ce07a629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:27.258646 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffccf81-9153-47f2-832d-d7a5ce07a629 admin admin] PATH_INFO: `/v3/projects/a9bbff1a84a34de580f0062eb0e41793` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:27.259893 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffccf81-9153-47f2-832d-d7a5ce07a629 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9bbff1a84a34de580f0062eb0e41793)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:27.262829 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffccf81-9153-47f2-832d-d7a5ce07a629 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:27.325996 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-fffccf81-9153-47f2-832d-d7a5ce07a629 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9bbff1a84a34de580f0062eb0e41793', 'request_id': 'req-fffccf81-9153-47f2-832d-d7a5ce07a629'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:27.335173 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-fffccf81-9153-47f2-832d-d7a5ce07a629 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9bbff1a84a34de580f0062eb0e41793', 'request_id': 'req-fffccf81-9153-47f2-832d-d7a5ce07a629'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:27.342021 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-fffccf81-9153-47f2-832d-d7a5ce07a629 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9bbff1a84a34de580f0062eb0e41793', 'request_id': 'req-fffccf81-9153-47f2-832d-d7a5ce07a629'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:27.356272 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-fffccf81-9153-47f2-832d-d7a5ce07a629 admin admin] The token cache is being invalidate because project a9bbff1a84a34de580f0062eb0e41793 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:52:27.356748 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-fffccf81-9153-47f2-832d-d7a5ce07a629 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:27.357562 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-fffccf81-9153-47f2-832d-d7a5ce07a629 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:27.359345 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1420/2865] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:52:27 2022] DELETE /identity/v3/projects/a9bbff1a84a34de580f0062eb0e41793 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:52:27.363394 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73622ea6-8c49-4c92-a5c2-1d2a39821cdb service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:27.377967 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73622ea6-8c49-4c92-a5c2-1d2a39821cdb service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:27.441546 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c35d6b5-182d-4212-b42f-58d361b915d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:27.442462 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35d6b5-182d-4212-b42f-58d361b915d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:27.442747 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35d6b5-182d-4212-b42f-58d361b915d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:27.443026 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35d6b5-182d-4212-b42f-58d361b915d4 admin admin] PATH_INFO: `/v3/users/777b53b121dd4410b9d39654eb07b53b` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:27.444960 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c35d6b5-182d-4212-b42f-58d361b915d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=777b53b121dd4410b9d39654eb07b53b)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:27.446969 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c35d6b5-182d-4212-b42f-58d361b915d4 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:27.507332 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-4c35d6b5-182d-4212-b42f-58d361b915d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '777b53b121dd4410b9d39654eb07b53b'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:27.523748 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:52:27.536524 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-4c35d6b5-182d-4212-b42f-58d361b915d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '777b53b121dd4410b9d39654eb07b53b'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:27.552974 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-4c35d6b5-182d-4212-b42f-58d361b915d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '777b53b121dd4410b9d39654eb07b53b'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:27.554599 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-4c35d6b5-182d-4212-b42f-58d361b915d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '777b53b121dd4410b9d39654eb07b53b'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:27.555613 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-4c35d6b5-182d-4212-b42f-58d361b915d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '777b53b121dd4410b9d39654eb07b53b'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:27.574174 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1446/2866] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:52:27 2022] DELETE /identity/v3/users/777b53b121dd4410b9d39654eb07b53b => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:52:28.039730 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffccf81-9153-47f2-832d-d7a5ce07a629 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:28.041223 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-fffccf81-9153-47f2-832d-d7a5ce07a629 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:28.071613 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffccf81-9153-47f2-832d-d7a5ce07a629 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:28.165456 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9048ded-2f12-4050-8249-c7736f23d893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:28.166803 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9048ded-2f12-4050-8249-c7736f23d893 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:28.166803 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9048ded-2f12-4050-8249-c7736f23d893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:28.167308 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9048ded-2f12-4050-8249-c7736f23d893 admin admin] PATH_INFO: `/v3/projects/414897f0cb01474583918890223b7c9a` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:28.168717 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9048ded-2f12-4050-8249-c7736f23d893 admin admin] RBAC: Authorizing `identity:delete_project(project_id=414897f0cb01474583918890223b7c9a)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:28.170888 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9048ded-2f12-4050-8249-c7736f23d893 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:28.267305 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f9048ded-2f12-4050-8249-c7736f23d893 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '414897f0cb01474583918890223b7c9a', 'request_id': 'req-f9048ded-2f12-4050-8249-c7736f23d893'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:28.281095 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f9048ded-2f12-4050-8249-c7736f23d893 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '414897f0cb01474583918890223b7c9a', 'request_id': 'req-f9048ded-2f12-4050-8249-c7736f23d893'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:28.293026 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f9048ded-2f12-4050-8249-c7736f23d893 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '414897f0cb01474583918890223b7c9a', 'request_id': 'req-f9048ded-2f12-4050-8249-c7736f23d893'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:28.316081 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f9048ded-2f12-4050-8249-c7736f23d893 admin admin] The token cache is being invalidate because project 414897f0cb01474583918890223b7c9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:52:28.316778 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f9048ded-2f12-4050-8249-c7736f23d893 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:28.318154 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f9048ded-2f12-4050-8249-c7736f23d893 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:52:28.324913 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1421/2867] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:52:28 2022] DELETE /identity/v3/projects/414897f0cb01474583918890223b7c9a => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:52:28.327035 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af674f-575c-4d53-8a49-cf9efec3cb69 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:28.328207 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af674f-575c-4d53-8a49-cf9efec3cb69 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:28.328498 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af674f-575c-4d53-8a49-cf9efec3cb69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:28.352913 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-f5af674f-575c-4d53-8a49-cf9efec3cb69 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:52:28.354250 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-f5af674f-575c-4d53-8a49-cf9efec3cb69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:28.424641 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1447/2868] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:52:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:52:28.434701 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e809d17-1425-4a09-b4af-0349cc495e99 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:28.435067 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e809d17-1425-4a09-b4af-0349cc495e99 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:28.435341 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e809d17-1425-4a09-b4af-0349cc495e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:28.455119 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-5e809d17-1425-4a09-b4af-0349cc495e99 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:52:28.456477 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-5e809d17-1425-4a09-b4af-0349cc495e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:28.465858 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1422/2869] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:52:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:52:28.470152 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5af674f-575c-4d53-8a49-cf9efec3cb69 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:28.480562 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5af674f-575c-4d53-8a49-cf9efec3cb69 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:28.485598 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3fbe83-d207-43c9-b31f-be26d32ad5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:28.486430 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3fbe83-d207-43c9-b31f-be26d32ad5da admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:28.486781 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3fbe83-d207-43c9-b31f-be26d32ad5da admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:28.487123 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3fbe83-d207-43c9-b31f-be26d32ad5da admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:28.488026 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3fbe83-d207-43c9-b31f-be26d32ad5da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:52:28.488396 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3fbe83-d207-43c9-b31f-be26d32ad5da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:28.490105 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3fbe83-d207-43c9-b31f-be26d32ad5da admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:28.503521 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1448/2870] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:52:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:52:28.508117 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e809d17-1425-4a09-b4af-0349cc495e99 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:28.513958 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:52:28.518040 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e809d17-1425-4a09-b4af-0349cc495e99 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:28.525019 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fb41aa-8002-4d91-a78e-aa7ff599cabf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:28.525785 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fb41aa-8002-4d91-a78e-aa7ff599cabf admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:28.526152 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fb41aa-8002-4d91-a78e-aa7ff599cabf admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:28.526600 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fb41aa-8002-4d91-a78e-aa7ff599cabf admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:28.527485 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fb41aa-8002-4d91-a78e-aa7ff599cabf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:28.529610 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fb41aa-8002-4d91-a78e-aa7ff599cabf admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:28.561268 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1423/2871] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:52:28 2022] POST /identity/v3/projects => generated 367 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:52:28.562714 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3fbe83-d207-43c9-b31f-be26d32ad5da admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:28.592484 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3fbe83-d207-43c9-b31f-be26d32ad5da admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:28.655650 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f323b7-5f63-4828-af63-e52088efd20e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:28.656254 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f323b7-5f63-4828-af63-e52088efd20e admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:28.656543 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f323b7-5f63-4828-af63-e52088efd20e admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:28.656822 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f323b7-5f63-4828-af63-e52088efd20e admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:28.657550 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f323b7-5f63-4828-af63-e52088efd20e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:28.659604 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f323b7-5f63-4828-af63-e52088efd20e admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:28.693495 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1449/2872] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:52:28 2022] POST /identity/v3/users => generated 284 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:52:28.698992 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fb41aa-8002-4d91-a78e-aa7ff599cabf admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:28.719377 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fb41aa-8002-4d91-a78e-aa7ff599cabf admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:28.724182 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683d847d-32e1-429d-8494-26e5c8381f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:28.724856 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683d847d-32e1-429d-8494-26e5c8381f78 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:28.725735 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683d847d-32e1-429d-8494-26e5c8381f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:28.726753 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683d847d-32e1-429d-8494-26e5c8381f78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:28.727507 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683d847d-32e1-429d-8494-26e5c8381f78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:28.729134 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683d847d-32e1-429d-8494-26e5c8381f78 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:28.743269 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1424/2873] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:52:28.747154 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f323b7-5f63-4828-af63-e52088efd20e admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:28.758375 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f323b7-5f63-4828-af63-e52088efd20e admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:28.763580 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b44d9d1-55df-4fe7-8512-cbb24e1ade52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:28.764257 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b44d9d1-55df-4fe7-8512-cbb24e1ade52 admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:28.764541 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b44d9d1-55df-4fe7-8512-cbb24e1ade52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:28.764817 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b44d9d1-55df-4fe7-8512-cbb24e1ade52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:28.765551 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b44d9d1-55df-4fe7-8512-cbb24e1ade52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:28.767219 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b44d9d1-55df-4fe7-8512-cbb24e1ade52 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:28.778647 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1450/2874] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:52:28.782961 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683d847d-32e1-429d-8494-26e5c8381f78 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:28.794706 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683d847d-32e1-429d-8494-26e5c8381f78 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:28.799473 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dfdad6-ca49-46ca-89d3-7bf324ad98d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:28.800363 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dfdad6-ca49-46ca-89d3-7bf324ad98d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:28.800643 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dfdad6-ca49-46ca-89d3-7bf324ad98d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:28.800916 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dfdad6-ca49-46ca-89d3-7bf324ad98d6 admin admin] PATH_INFO: `/v3/projects/93103876445049599abeb8112c1ad33c/users/91724b80692b4488be0f96e677d4ca79/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:28.817894 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dfdad6-ca49-46ca-89d3-7bf324ad98d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93103876445049599abeb8112c1ad33c, user_id=91724b80692b4488be0f96e677d4ca79, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:28.820756 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dfdad6-ca49-46ca-89d3-7bf324ad98d6 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:28.834720 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1425/2875] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:52:28 2022] PUT /identity/v3/projects/93103876445049599abeb8112c1ad33c/users/91724b80692b4488be0f96e677d4ca79/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:52:28.846325 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7149fdd-a464-4f60-bbf2-c3d5a505f170 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:28.846637 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7149fdd-a464-4f60-bbf2-c3d5a505f170 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:28.846912 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7149fdd-a464-4f60-bbf2-c3d5a505f170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:28.876058 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-d7149fdd-a464-4f60-bbf2-c3d5a505f170 None None] MFA Rules not processed for user `91724b80692b4488be0f96e677d4ca79`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:52:28.877274 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-d7149fdd-a464-4f60-bbf2-c3d5a505f170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:28.915074 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:52:28.964315 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1451/2876] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:52:28 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:52:28.975926 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dfdad6-ca49-46ca-89d3-7bf324ad98d6 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:28.977618 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-23dfdad6-ca49-46ca-89d3-7bf324ad98d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:29.040795 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dfdad6-ca49-46ca-89d3-7bf324ad98d6 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:29.044521 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c0a894-b930-4e9c-aecd-ca7a8e97242d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd15c5f6aeb874191b73c8ee0410c6bc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:29.045175 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0a894-b930-4e9c-aecd-ca7a8e97242d service glance] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:29.045461 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0a894-b930-4e9c-aecd-ca7a8e97242d service glance] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:29.045768 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0a894-b930-4e9c-aecd-ca7a8e97242d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:29.056798 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c0a894-b930-4e9c-aecd-ca7a8e97242d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:29.058634 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c0a894-b930-4e9c-aecd-ca7a8e97242d service glance] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:29.068628 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1426/2877] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:52:28 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:52:38.964377 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=15 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:52:39.064222 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=48,keystone:INSERT=2 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:53:00.574256 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7149fdd-a464-4f60-bbf2-c3d5a505f170 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:53:00.678526 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7149fdd-a464-4f60-bbf2-c3d5a505f170 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:53:00.683107 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68110e53-ba0d-4734-a7cf-01beaf81abc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:53:00.683978 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68110e53-ba0d-4734-a7cf-01beaf81abc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:53:00.684262 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68110e53-ba0d-4734-a7cf-01beaf81abc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:53:00.684534 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68110e53-ba0d-4734-a7cf-01beaf81abc2 admin admin] PATH_INFO: `/v3/users/91724b80692b4488be0f96e677d4ca79` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:53:00.685732 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68110e53-ba0d-4734-a7cf-01beaf81abc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91724b80692b4488be0f96e677d4ca79)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:53:00.687661 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68110e53-ba0d-4734-a7cf-01beaf81abc2 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:53:00.735307 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-68110e53-ba0d-4734-a7cf-01beaf81abc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91724b80692b4488be0f96e677d4ca79'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:53:00.747724 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-68110e53-ba0d-4734-a7cf-01beaf81abc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91724b80692b4488be0f96e677d4ca79'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:53:00.756411 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-68110e53-ba0d-4734-a7cf-01beaf81abc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91724b80692b4488be0f96e677d4ca79'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:53:00.757061 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-68110e53-ba0d-4734-a7cf-01beaf81abc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91724b80692b4488be0f96e677d4ca79'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:53:00.757662 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-68110e53-ba0d-4734-a7cf-01beaf81abc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91724b80692b4488be0f96e677d4ca79'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:53:00.769958 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1452/2878] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:53:00 2022] DELETE /identity/v3/users/91724b80692b4488be0f96e677d4ca79 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:53:00.775399 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c90e4-b087-4067-a124-c171932b38fd None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:53:00.775890 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c90e4-b087-4067-a124-c171932b38fd None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:53:00.775989 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c90e4-b087-4067-a124-c171932b38fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:53:00.816605 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-c65c90e4-b087-4067-a124-c171932b38fd None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:53:00.819008 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-c65c90e4-b087-4067-a124-c171932b38fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:53:00.878095 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1427/2879] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:53:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:53:00.889554 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68110e53-ba0d-4734-a7cf-01beaf81abc2 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:53:00.891023 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-68110e53-ba0d-4734-a7cf-01beaf81abc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:53:00.935732 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:53:00.973100 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68110e53-ba0d-4734-a7cf-01beaf81abc2 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:53:00.977269 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80902c1-2da3-4525-8579-3131b591a6bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:53:00.977988 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80902c1-2da3-4525-8579-3131b591a6bf service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:53:00.978271 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80902c1-2da3-4525-8579-3131b591a6bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:53:00.978547 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80902c1-2da3-4525-8579-3131b591a6bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:53:00.987859 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80902c1-2da3-4525-8579-3131b591a6bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:53:00.989917 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80902c1-2da3-4525-8579-3131b591a6bf service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:53:00.999857 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1453/2880] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:53:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:53:01.293990 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65c90e4-b087-4067-a124-c171932b38fd None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:53:01.296145 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-c65c90e4-b087-4067-a124-c171932b38fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:53:01.310264 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65c90e4-b087-4067-a124-c171932b38fd None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:53:01.315635 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c608fda-12f2-4b7b-8681-0f69bb6a74ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:53:01.316511 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c608fda-12f2-4b7b-8681-0f69bb6a74ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:53:01.316805 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c608fda-12f2-4b7b-8681-0f69bb6a74ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:53:01.317081 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c608fda-12f2-4b7b-8681-0f69bb6a74ca admin admin] PATH_INFO: `/v3/projects/93103876445049599abeb8112c1ad33c` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:53:01.319101 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c608fda-12f2-4b7b-8681-0f69bb6a74ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=93103876445049599abeb8112c1ad33c)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:53:01.321247 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c608fda-12f2-4b7b-8681-0f69bb6a74ca admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:53:01.373273 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-3c608fda-12f2-4b7b-8681-0f69bb6a74ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93103876445049599abeb8112c1ad33c', 'request_id': 'req-3c608fda-12f2-4b7b-8681-0f69bb6a74ca'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:53:01.380890 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-3c608fda-12f2-4b7b-8681-0f69bb6a74ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93103876445049599abeb8112c1ad33c', 'request_id': 'req-3c608fda-12f2-4b7b-8681-0f69bb6a74ca'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:53:01.387268 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-3c608fda-12f2-4b7b-8681-0f69bb6a74ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93103876445049599abeb8112c1ad33c', 'request_id': 'req-3c608fda-12f2-4b7b-8681-0f69bb6a74ca'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:53:01.402815 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-3c608fda-12f2-4b7b-8681-0f69bb6a74ca admin admin] The token cache is being invalidate because project 93103876445049599abeb8112c1ad33c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:53:01.402815 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-3c608fda-12f2-4b7b-8681-0f69bb6a74ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:53:01.403505 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-3c608fda-12f2-4b7b-8681-0f69bb6a74ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:53:01.405319 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1428/2881] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:53:01 2022] DELETE /identity/v3/projects/93103876445049599abeb8112c1ad33c => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:53:01.411560 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0793ccf-dc68-4fe4-86f8-577e83f7d6af None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:53:01.412180 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0793ccf-dc68-4fe4-86f8-577e83f7d6af None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:53:01.412554 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0793ccf-dc68-4fe4-86f8-577e83f7d6af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:53:01.432741 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-e0793ccf-dc68-4fe4-86f8-577e83f7d6af None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:53:01.433980 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-e0793ccf-dc68-4fe4-86f8-577e83f7d6af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:53:01.498650 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1454/2882] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:53:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:53:01.507368 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e10c6cd-02e1-42ea-a76c-2eb1d3b2b9b5 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:53:01.507652 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e10c6cd-02e1-42ea-a76c-2eb1d3b2b9b5 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:53:01.507926 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e10c6cd-02e1-42ea-a76c-2eb1d3b2b9b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:53:01.526178 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-1e10c6cd-02e1-42ea-a76c-2eb1d3b2b9b5 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:53:01.527388 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-1e10c6cd-02e1-42ea-a76c-2eb1d3b2b9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:53:01.535371 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1429/2883] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:53:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:53:01.539830 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0793ccf-dc68-4fe4-86f8-577e83f7d6af None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:53:01.549950 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0793ccf-dc68-4fe4-86f8-577e83f7d6af None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:53:01.555349 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68928e7-8126-4ca5-98c7-1368d0f8155a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:53:01.556014 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68928e7-8126-4ca5-98c7-1368d0f8155a admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:53:01.556323 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68928e7-8126-4ca5-98c7-1368d0f8155a admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:53:01.556615 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68928e7-8126-4ca5-98c7-1368d0f8155a admin admin] PATH_INFO: `/v3/domains` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:53:01.557347 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68928e7-8126-4ca5-98c7-1368d0f8155a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:53:01.557667 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68928e7-8126-4ca5-98c7-1368d0f8155a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:53:01.559297 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68928e7-8126-4ca5-98c7-1368d0f8155a admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:53:01.572385 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1455/2884] 10.222.0.13 () {58 vars in 1152 bytes} [Fri Jun 24 08:53:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:53:01.576930 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e10c6cd-02e1-42ea-a76c-2eb1d3b2b9b5 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:53:01.586542 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e10c6cd-02e1-42ea-a76c-2eb1d3b2b9b5 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:53:01.591865 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ea3ea-1c08-43d7-831e-ef7417032b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:53:01.592547 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ea3ea-1c08-43d7-831e-ef7417032b7b admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:53:01.592832 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ea3ea-1c08-43d7-831e-ef7417032b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:53:01.593110 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ea3ea-1c08-43d7-831e-ef7417032b7b admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:53:01.593840 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519ea3ea-1c08-43d7-831e-ef7417032b7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:53:01.595757 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519ea3ea-1c08-43d7-831e-ef7417032b7b admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:53:01.622089 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1430/2885] 10.222.0.13 () {60 vars in 1152 bytes} [Fri Jun 24 08:53:01 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:53:01.626364 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68928e7-8126-4ca5-98c7-1368d0f8155a admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:53:01.638147 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68928e7-8126-4ca5-98c7-1368d0f8155a admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:53:01.696254 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3766265a-effa-4d9c-87b6-89b0fff351a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:53:01.696923 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3766265a-effa-4d9c-87b6-89b0fff351a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:53:01.697206 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3766265a-effa-4d9c-87b6-89b0fff351a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:53:01.697509 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3766265a-effa-4d9c-87b6-89b0fff351a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:53:01.698274 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3766265a-effa-4d9c-87b6-89b0fff351a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:53:01.700178 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3766265a-effa-4d9c-87b6-89b0fff351a8 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:53:01.720161 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1456/2886] 10.222.0.13 () {60 vars in 1143 bytes} [Fri Jun 24 08:53:01 2022] POST /identity/v3/users => generated 285 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:53:01.724998 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ea3ea-1c08-43d7-831e-ef7417032b7b admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:53:01.736235 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ea3ea-1c08-43d7-831e-ef7417032b7b admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:53:01.742246 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f8ce8d-f926-4b96-b6ff-257c0f259528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:53:01.742915 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f8ce8d-f926-4b96-b6ff-257c0f259528 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:53:01.743266 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f8ce8d-f926-4b96-b6ff-257c0f259528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:53:01.743533 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f8ce8d-f926-4b96-b6ff-257c0f259528 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:53:01.744259 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f8ce8d-f926-4b96-b6ff-257c0f259528 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:53:01.745897 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f8ce8d-f926-4b96-b6ff-257c0f259528 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:53:01.757414 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1431/2887] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:53:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:53:01.761497 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3766265a-effa-4d9c-87b6-89b0fff351a8 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:53:01.771914 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3766265a-effa-4d9c-87b6-89b0fff351a8 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:53:01.776717 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d011a5b-ef80-47c1-95b4-d3407b489d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:53:01.777376 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d011a5b-ef80-47c1-95b4-d3407b489d1c admin admin] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:53:01.777663 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d011a5b-ef80-47c1-95b4-d3407b489d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:53:01.777972 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d011a5b-ef80-47c1-95b4-d3407b489d1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:53:01.778702 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d011a5b-ef80-47c1-95b4-d3407b489d1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:53:01.780304 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d011a5b-ef80-47c1-95b4-d3407b489d1c admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:53:01.790923 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1457/2888] 10.222.0.13 () {58 vars in 1121 bytes} [Fri Jun 24 08:53:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:53:01.795177 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f8ce8d-f926-4b96-b6ff-257c0f259528 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:53:01.806383 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f8ce8d-f926-4b96-b6ff-257c0f259528 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:53:01.811754 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86e8c06-b04b-4a14-a588-58d168789f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:53:01.812647 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e8c06-b04b-4a14-a588-58d168789f26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:53:01.812929 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e8c06-b04b-4a14-a588-58d168789f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:53:01.813205 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e8c06-b04b-4a14-a588-58d168789f26 admin admin] PATH_INFO: `/v3/projects/b718196f73544c1895fe664184400faa/users/cce70dfa26ef4bcf9f0fa82e1177b702/roles/bd56838b71ec4a5781fe2df9f3f4f1ab` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:53:01.828365 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86e8c06-b04b-4a14-a588-58d168789f26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b718196f73544c1895fe664184400faa, user_id=cce70dfa26ef4bcf9f0fa82e1177b702, role_id=bd56838b71ec4a5781fe2df9f3f4f1ab)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:53:01.831082 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86e8c06-b04b-4a14-a588-58d168789f26 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:53:01.840283 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:53:01.845868 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1432/2889] 10.222.0.13 () {60 vars in 1482 bytes} [Fri Jun 24 08:53:01 2022] PUT /identity/v3/projects/b718196f73544c1895fe664184400faa/users/cce70dfa26ef4bcf9f0fa82e1177b702/roles/bd56838b71ec4a5781fe2df9f3f4f1ab => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:53:01.857675 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce513c4-6c53-4b70-aa11-60dbdcd0c491 None None] REQUEST_METHOD: `POST` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:53:01.858001 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce513c4-6c53-4b70-aa11-60dbdcd0c491 None None] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:53:01.858282 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce513c4-6c53-4b70-aa11-60dbdcd0c491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:53:01.885438 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.auth.core [None req-cce513c4-6c53-4b70-aa11-60dbdcd0c491 None None] MFA Rules not processed for user `cce70dfa26ef4bcf9f0fa82e1177b702`. Rule list: `[]` (Enabled: `True`). {{(pid=80424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:53:01.886701 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-cce513c4-6c53-4b70-aa11-60dbdcd0c491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:53:01.906559 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:53:01.957130 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1458/2890] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:53:01 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:53:01.968498 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86e8c06-b04b-4a14-a588-58d168789f26 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:53:01.970248 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-c86e8c06-b04b-4a14-a588-58d168789f26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:53:02.032200 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86e8c06-b04b-4a14-a588-58d168789f26 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:53:02.036739 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db74ac74-1eff-460a-9335-acf48600c7ab service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd15c5f6aeb874191b73c8ee0410c6bc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:53:02.037474 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db74ac74-1eff-460a-9335-acf48600c7ab service glance] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:53:02.037780 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db74ac74-1eff-460a-9335-acf48600c7ab service glance] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:53:02.038188 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db74ac74-1eff-460a-9335-acf48600c7ab service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:53:02.046952 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db74ac74-1eff-460a-9335-acf48600c7ab service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:53:02.048389 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db74ac74-1eff-460a-9335-acf48600c7ab service glance] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:53:02.057888 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1433/2891] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:53:01 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:53:11.954098 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=18 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:53:12.053368 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=26 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:53:27.861967 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce513c4-6c53-4b70-aa11-60dbdcd0c491 None None] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:53:27.943749 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce513c4-6c53-4b70-aa11-60dbdcd0c491 None None] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:53:27.948644 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5afe20-c322-4644-b92a-775a3dcd8afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:53:27.949545 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5afe20-c322-4644-b92a-775a3dcd8afc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:53:27.949866 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5afe20-c322-4644-b92a-775a3dcd8afc admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:53:27.950148 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5afe20-c322-4644-b92a-775a3dcd8afc admin admin] PATH_INFO: `/v3/users/cce70dfa26ef4bcf9f0fa82e1177b702` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:53:27.951429 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5afe20-c322-4644-b92a-775a3dcd8afc admin admin] RBAC: Authorizing `identity:delete_user(user_id=cce70dfa26ef4bcf9f0fa82e1177b702)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:53:27.953426 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5afe20-c322-4644-b92a-775a3dcd8afc admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:53:28.007312 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-1a5afe20-c322-4644-b92a-775a3dcd8afc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cce70dfa26ef4bcf9f0fa82e1177b702'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:53:28.020285 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-1a5afe20-c322-4644-b92a-775a3dcd8afc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cce70dfa26ef4bcf9f0fa82e1177b702'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:53:28.032414 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-1a5afe20-c322-4644-b92a-775a3dcd8afc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cce70dfa26ef4bcf9f0fa82e1177b702'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:53:28.033053 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-1a5afe20-c322-4644-b92a-775a3dcd8afc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cce70dfa26ef4bcf9f0fa82e1177b702'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:53:28.033667 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-1a5afe20-c322-4644-b92a-775a3dcd8afc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cce70dfa26ef4bcf9f0fa82e1177b702'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:53:28.048080 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1459/2892] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:53:27 2022] DELETE /identity/v3/users/cce70dfa26ef4bcf9f0fa82e1177b702 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:53:28.054143 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfdd96e-9795-4512-825f-fbafede41490 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:53:28.054693 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfdd96e-9795-4512-825f-fbafede41490 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:53:28.054693 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfdd96e-9795-4512-825f-fbafede41490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:53:28.085013 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-cbfdd96e-9795-4512-825f-fbafede41490 None None] MFA Rules not processed for user `268e75b173ff435b8dfbe2517f69c015`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:53:28.086702 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-cbfdd96e-9795-4512-825f-fbafede41490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:53:28.149432 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1434/2893] 10.222.0.13 () {58 vars in 957 bytes} [Fri Jun 24 08:53:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:53:28.161112 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5afe20-c322-4644-b92a-775a3dcd8afc admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:53:28.162585 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-1a5afe20-c322-4644-b92a-775a3dcd8afc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:53:28.249473 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5afe20-c322-4644-b92a-775a3dcd8afc admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:53:28.253278 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8b18e2-3821-44d9-b9ca-8f62eaedc589 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:53:28.254239 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b18e2-3821-44d9-b9ca-8f62eaedc589 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:53:28.254518 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b18e2-3821-44d9-b9ca-8f62eaedc589 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:53:28.254792 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b18e2-3821-44d9-b9ca-8f62eaedc589 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:53:28.264510 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8b18e2-3821-44d9-b9ca-8f62eaedc589 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:53:28.265983 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8b18e2-3821-44d9-b9ca-8f62eaedc589 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:53:28.276415 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1460/2894] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:53:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:53:28.608418 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfdd96e-9795-4512-825f-fbafede41490 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:53:28.610803 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-cbfdd96e-9795-4512-825f-fbafede41490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:53:28.632659 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfdd96e-9795-4512-825f-fbafede41490 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:53:28.638251 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a19cff8-0cfe-4d02-9f5a-21d71af49ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:53:28.638986 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a19cff8-0cfe-4d02-9f5a-21d71af49ec2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:53:28.639270 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a19cff8-0cfe-4d02-9f5a-21d71af49ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:53:28.639607 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a19cff8-0cfe-4d02-9f5a-21d71af49ec2 admin admin] PATH_INFO: `/v3/projects/b718196f73544c1895fe664184400faa` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:53:28.640902 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a19cff8-0cfe-4d02-9f5a-21d71af49ec2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b718196f73544c1895fe664184400faa)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:53:28.642953 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a19cff8-0cfe-4d02-9f5a-21d71af49ec2 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:53:28.707265 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-8a19cff8-0cfe-4d02-9f5a-21d71af49ec2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b718196f73544c1895fe664184400faa', 'request_id': 'req-8a19cff8-0cfe-4d02-9f5a-21d71af49ec2'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:53:28.714274 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-8a19cff8-0cfe-4d02-9f5a-21d71af49ec2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b718196f73544c1895fe664184400faa', 'request_id': 'req-8a19cff8-0cfe-4d02-9f5a-21d71af49ec2'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:53:28.720749 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-8a19cff8-0cfe-4d02-9f5a-21d71af49ec2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b718196f73544c1895fe664184400faa', 'request_id': 'req-8a19cff8-0cfe-4d02-9f5a-21d71af49ec2'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:53:28.734931 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-8a19cff8-0cfe-4d02-9f5a-21d71af49ec2 admin admin] The token cache is being invalidate because project b718196f73544c1895fe664184400faa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:53:28.735221 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-8a19cff8-0cfe-4d02-9f5a-21d71af49ec2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:53:28.735861 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-8a19cff8-0cfe-4d02-9f5a-21d71af49ec2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:53:28.737540 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1435/2895] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:53:28 2022] DELETE /identity/v3/projects/b718196f73544c1895fe664184400faa => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:53:38.271940 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:53:38.731626 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:54:45.820935 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8b18e2-3821-44d9-b9ca-8f62eaedc589 service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:54:45.822378 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-7e8b18e2-3821-44d9-b9ca-8f62eaedc589 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:54:45.914036 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8b18e2-3821-44d9-b9ca-8f62eaedc589 service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:54:45.976782 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c13a7d-1bae-424a-a798-0ffcbf3a54a9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39cc268bad604dd08594e3b39c7ecf5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:54:45.977437 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c13a7d-1bae-424a-a798-0ffcbf3a54a9 service placement] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:54:45.977754 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c13a7d-1bae-424a-a798-0ffcbf3a54a9 service placement] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:54:45.978040 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c13a7d-1bae-424a-a798-0ffcbf3a54a9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:54:45.979843 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-43c13a7d-1bae-424a-a798-0ffcbf3a54a9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:54:45.992957 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c13a7d-1bae-424a-a798-0ffcbf3a54a9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:54:45.994595 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c13a7d-1bae-424a-a798-0ffcbf3a54a9 service placement] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:54:46.005431 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1461/2896] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:54:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:54:46.561324 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a19cff8-0cfe-4d02-9f5a-21d71af49ec2 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:54:46.563273 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-8a19cff8-0cfe-4d02-9f5a-21d71af49ec2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:54:46.635106 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a19cff8-0cfe-4d02-9f5a-21d71af49ec2 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:54:46.639434 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbe7af8-0145-4c8d-a4e9-aeee2afa378f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:54:46.640354 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbe7af8-0145-4c8d-a4e9-aeee2afa378f service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:54:46.641190 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbe7af8-0145-4c8d-a4e9-aeee2afa378f service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:54:46.641453 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbe7af8-0145-4c8d-a4e9-aeee2afa378f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:54:46.644478 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-adbe7af8-0145-4c8d-a4e9-aeee2afa378f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:54:46.655769 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbe7af8-0145-4c8d-a4e9-aeee2afa378f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:54:46.658085 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbe7af8-0145-4c8d-a4e9-aeee2afa378f service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:54:46.668359 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1436/2897] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:54:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:54:46.767345 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c13a7d-1bae-424a-a798-0ffcbf3a54a9 service placement] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:54:46.777348 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c13a7d-1bae-424a-a798-0ffcbf3a54a9 service placement] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:54:46.781617 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdb88e3-d977-45d8-b14e-de7ce4175660 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:54:46.782522 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb88e3-d977-45d8-b14e-de7ce4175660 service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:54:46.782935 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb88e3-d977-45d8-b14e-de7ce4175660 service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:54:46.783289 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb88e3-d977-45d8-b14e-de7ce4175660 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:54:46.785199 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-5fdb88e3-d977-45d8-b14e-de7ce4175660 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:54:46.857478 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=100 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:54:46.882523 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdb88e3-d977-45d8-b14e-de7ce4175660 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:54:46.883957 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdb88e3-d977-45d8-b14e-de7ce4175660 service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:54:46.907300 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1462/2898] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:54:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:54:56.664088 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=26 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:54:56.903610 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=13 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:56:14.803521 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbe7af8-0145-4c8d-a4e9-aeee2afa378f service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:14.805104 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-adbe7af8-0145-4c8d-a4e9-aeee2afa378f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:56:14.825010 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbe7af8-0145-4c8d-a4e9-aeee2afa378f service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:14.831939 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e27798e-ee9e-46de-a081-cb514c53a5e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74bb1bc6821b4500b26f160ccae5b7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:14.832800 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e27798e-ee9e-46de-a081-cb514c53a5e9 service nova] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:14.833156 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e27798e-ee9e-46de-a081-cb514c53a5e9 service nova] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:14.833508 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e27798e-ee9e-46de-a081-cb514c53a5e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:14.835480 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-0e27798e-ee9e-46de-a081-cb514c53a5e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:56:14.851416 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e27798e-ee9e-46de-a081-cb514c53a5e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:14.853019 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e27798e-ee9e-46de-a081-cb514c53a5e9 service nova] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:14.866139 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1437/2899] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:56:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:56:14.953802 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdb88e3-d977-45d8-b14e-de7ce4175660 service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:14.966365 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdb88e3-d977-45d8-b14e-de7ce4175660 service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:14.970867 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e419e-091a-46a9-8708-b8606f44072c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:14.971641 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e419e-091a-46a9-8708-b8606f44072c service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:14.972058 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e419e-091a-46a9-8708-b8606f44072c service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:14.972405 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e419e-091a-46a9-8708-b8606f44072c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:14.974404 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-b27e419e-091a-46a9-8708-b8606f44072c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:56:14.988599 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27e419e-091a-46a9-8708-b8606f44072c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:14.990383 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27e419e-091a-46a9-8708-b8606f44072c service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:15.004769 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1463/2900] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:56:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:56:15.044077 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e27798e-ee9e-46de-a081-cb514c53a5e9 service nova] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:15.056364 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e27798e-ee9e-46de-a081-cb514c53a5e9 service nova] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:15.060896 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccf6c5e-fab5-4c9f-a7d4-15b6116b16fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:15.061547 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf6c5e-fab5-4c9f-a7d4-15b6116b16fe service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:15.062111 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf6c5e-fab5-4c9f-a7d4-15b6116b16fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:15.062418 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf6c5e-fab5-4c9f-a7d4-15b6116b16fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:15.064224 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-eccf6c5e-fab5-4c9f-a7d4-15b6116b16fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:56:15.079170 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccf6c5e-fab5-4c9f-a7d4-15b6116b16fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:15.080646 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccf6c5e-fab5-4c9f-a7d4-15b6116b16fe service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:15.092678 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1438/2901] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:56:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:56:16.279079 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e419e-091a-46a9-8708-b8606f44072c service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:16.288160 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e419e-091a-46a9-8708-b8606f44072c service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:16.291913 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8efa218-cc8c-4969-bd9b-03003cd09aef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f7ba6c7faf43e7acbb43eee58d8c8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:16.292568 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8efa218-cc8c-4969-bd9b-03003cd09aef service neutron] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:16.292853 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8efa218-cc8c-4969-bd9b-03003cd09aef service neutron] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:16.293774 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8efa218-cc8c-4969-bd9b-03003cd09aef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:16.296403 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-f8efa218-cc8c-4969-bd9b-03003cd09aef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:56:16.311528 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8efa218-cc8c-4969-bd9b-03003cd09aef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:16.312991 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8efa218-cc8c-4969-bd9b-03003cd09aef service neutron] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:16.325313 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1464/2902] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:56:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:56:25.087279 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=12 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:56:25.395879 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccf6c5e-fab5-4c9f-a7d4-15b6116b16fe service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:25.397276 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-eccf6c5e-fab5-4c9f-a7d4-15b6116b16fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:56:25.419534 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccf6c5e-fab5-4c9f-a7d4-15b6116b16fe service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:25.424615 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534e7907-ddaf-4802-a3ff-08b1bcf36b37 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39cc268bad604dd08594e3b39c7ecf5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:25.425266 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534e7907-ddaf-4802-a3ff-08b1bcf36b37 service placement] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:25.425556 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534e7907-ddaf-4802-a3ff-08b1bcf36b37 service placement] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:25.425934 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534e7907-ddaf-4802-a3ff-08b1bcf36b37 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:25.428188 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-534e7907-ddaf-4802-a3ff-08b1bcf36b37 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:56:25.442882 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534e7907-ddaf-4802-a3ff-08b1bcf36b37 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:25.446801 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534e7907-ddaf-4802-a3ff-08b1bcf36b37 service placement] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:25.456185 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8efa218-cc8c-4969-bd9b-03003cd09aef service neutron] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:25.458399 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-f8efa218-cc8c-4969-bd9b-03003cd09aef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:56:25.459783 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1439/2903] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:56:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:56:25.531431 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8efa218-cc8c-4969-bd9b-03003cd09aef service neutron] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:25.536128 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc5527d-378c-48a1-b613-7b15736ac899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:25.536952 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc5527d-378c-48a1-b613-7b15736ac899 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:25.537244 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc5527d-378c-48a1-b613-7b15736ac899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:25.537529 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc5527d-378c-48a1-b613-7b15736ac899 admin admin] PATH_INFO: `/v3/users/038c41319feb49a486832af853c332b1` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:25.538771 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc5527d-378c-48a1-b613-7b15736ac899 admin admin] RBAC: Authorizing `identity:delete_user(user_id=038c41319feb49a486832af853c332b1)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:25.540698 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc5527d-378c-48a1-b613-7b15736ac899 admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:25.589033 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-6dc5527d-378c-48a1-b613-7b15736ac899 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '038c41319feb49a486832af853c332b1'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:25.601002 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-6dc5527d-378c-48a1-b613-7b15736ac899 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '038c41319feb49a486832af853c332b1'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:25.609343 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-6dc5527d-378c-48a1-b613-7b15736ac899 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '038c41319feb49a486832af853c332b1'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:25.610108 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-6dc5527d-378c-48a1-b613-7b15736ac899 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '038c41319feb49a486832af853c332b1'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:25.610839 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-6dc5527d-378c-48a1-b613-7b15736ac899 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '038c41319feb49a486832af853c332b1'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:25.624117 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1465/2904] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:56:25 2022] DELETE /identity/v3/users/038c41319feb49a486832af853c332b1 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:56:25.809575 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534e7907-ddaf-4802-a3ff-08b1bcf36b37 service placement] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:25.826795 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534e7907-ddaf-4802-a3ff-08b1bcf36b37 service placement] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:25.883886 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20561cf-9bad-4621-b81d-ff51e34dfded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:25.884743 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20561cf-9bad-4621-b81d-ff51e34dfded admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:25.885025 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20561cf-9bad-4621-b81d-ff51e34dfded admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:25.885299 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20561cf-9bad-4621-b81d-ff51e34dfded admin admin] PATH_INFO: `/v3/projects/33c8998057e6449e9c881b1b279faadd` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:25.886521 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20561cf-9bad-4621-b81d-ff51e34dfded admin admin] RBAC: Authorizing `identity:delete_project(project_id=33c8998057e6449e9c881b1b279faadd)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:25.888422 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20561cf-9bad-4621-b81d-ff51e34dfded admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:25.934443 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-a20561cf-9bad-4621-b81d-ff51e34dfded admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33c8998057e6449e9c881b1b279faadd', 'request_id': 'req-a20561cf-9bad-4621-b81d-ff51e34dfded'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:25.940590 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-a20561cf-9bad-4621-b81d-ff51e34dfded admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33c8998057e6449e9c881b1b279faadd', 'request_id': 'req-a20561cf-9bad-4621-b81d-ff51e34dfded'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:25.946133 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-a20561cf-9bad-4621-b81d-ff51e34dfded admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33c8998057e6449e9c881b1b279faadd', 'request_id': 'req-a20561cf-9bad-4621-b81d-ff51e34dfded'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:25.959338 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-a20561cf-9bad-4621-b81d-ff51e34dfded admin admin] The token cache is being invalidate because project 33c8998057e6449e9c881b1b279faadd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:56:25.959628 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-a20561cf-9bad-4621-b81d-ff51e34dfded admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:25.960275 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-a20561cf-9bad-4621-b81d-ff51e34dfded admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:25.961882 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1440/2905] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:56:25 2022] DELETE /identity/v3/projects/33c8998057e6449e9c881b1b279faadd => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:56:25.966444 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc5527d-378c-48a1-b613-7b15736ac899 admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:25.967841 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-6dc5527d-378c-48a1-b613-7b15736ac899 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:56:25.983464 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc5527d-378c-48a1-b613-7b15736ac899 admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:26.035677 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:56:26.046964 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c33787-c43a-4d2e-a592-b1f90bdd1b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:26.047852 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c33787-c43a-4d2e-a592-b1f90bdd1b0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:26.048137 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c33787-c43a-4d2e-a592-b1f90bdd1b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:26.048412 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c33787-c43a-4d2e-a592-b1f90bdd1b0c admin admin] PATH_INFO: `/v3/users/1d6a85a232f047a69c6bdf5a19467988` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:26.049636 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c33787-c43a-4d2e-a592-b1f90bdd1b0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d6a85a232f047a69c6bdf5a19467988)` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:26.051639 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c33787-c43a-4d2e-a592-b1f90bdd1b0c admin admin] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:26.098455 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d3c33787-c43a-4d2e-a592-b1f90bdd1b0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d6a85a232f047a69c6bdf5a19467988'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:26.112152 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d3c33787-c43a-4d2e-a592-b1f90bdd1b0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d6a85a232f047a69c6bdf5a19467988'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:26.120536 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d3c33787-c43a-4d2e-a592-b1f90bdd1b0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d6a85a232f047a69c6bdf5a19467988'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:26.121793 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d3c33787-c43a-4d2e-a592-b1f90bdd1b0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d6a85a232f047a69c6bdf5a19467988'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:26.122408 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.notifications [None req-d3c33787-c43a-4d2e-a592-b1f90bdd1b0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d6a85a232f047a69c6bdf5a19467988'} {{(pid=80424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:26.134906 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1466/2906] 10.222.0.13 () {58 vars in 1223 bytes} [Fri Jun 24 08:56:25 2022] DELETE /identity/v3/users/1d6a85a232f047a69c6bdf5a19467988 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:56:26.311512 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20561cf-9bad-4621-b81d-ff51e34dfded admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:26.313065 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-a20561cf-9bad-4621-b81d-ff51e34dfded admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:56:26.333812 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20561cf-9bad-4621-b81d-ff51e34dfded admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:26.397239 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6cf4d-536f-4cd1-807d-ad0ad3ca3d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '268e75b173ff435b8dfbe2517f69c015', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '985b4af5ece540be8554aecad33b1dc4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:26.398140 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6cf4d-536f-4cd1-807d-ad0ad3ca3d4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:26.398427 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6cf4d-536f-4cd1-807d-ad0ad3ca3d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:26.398710 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6cf4d-536f-4cd1-807d-ad0ad3ca3d4f admin admin] PATH_INFO: `/v3/projects/c52ddf5ec3544b198de9b02b2cc49b03` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:26.399900 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb6cf4d-536f-4cd1-807d-ad0ad3ca3d4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c52ddf5ec3544b198de9b02b2cc49b03)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:26.401850 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb6cf4d-536f-4cd1-807d-ad0ad3ca3d4f admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:26.423400 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:56:26.450740 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-3fb6cf4d-536f-4cd1-807d-ad0ad3ca3d4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c52ddf5ec3544b198de9b02b2cc49b03', 'request_id': 'req-3fb6cf4d-536f-4cd1-807d-ad0ad3ca3d4f'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:26.457039 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-3fb6cf4d-536f-4cd1-807d-ad0ad3ca3d4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c52ddf5ec3544b198de9b02b2cc49b03', 'request_id': 'req-3fb6cf4d-536f-4cd1-807d-ad0ad3ca3d4f'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:26.462764 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-3fb6cf4d-536f-4cd1-807d-ad0ad3ca3d4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c52ddf5ec3544b198de9b02b2cc49b03', 'request_id': 'req-3fb6cf4d-536f-4cd1-807d-ad0ad3ca3d4f'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:26.477241 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-3fb6cf4d-536f-4cd1-807d-ad0ad3ca3d4f admin admin] The token cache is being invalidate because project c52ddf5ec3544b198de9b02b2cc49b03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:56:26.477540 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-3fb6cf4d-536f-4cd1-807d-ad0ad3ca3d4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:26.478201 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-3fb6cf4d-536f-4cd1-807d-ad0ad3ca3d4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:26.479756 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1441/2907] 10.222.0.13 () {58 vars in 1232 bytes} [Fri Jun 24 08:56:26 2022] DELETE /identity/v3/projects/c52ddf5ec3544b198de9b02b2cc49b03 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:56:36.131450 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:56:36.473814 dv-cinder-847131-2-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:59:49.552458 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c33787-c43a-4d2e-a592-b1f90bdd1b0c admin admin] Authenticating user token {{(pid=80424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:59:49.553973 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-d3c33787-c43a-4d2e-a592-b1f90bdd1b0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:59:49.632146 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c33787-c43a-4d2e-a592-b1f90bdd1b0c admin admin] Validating token access rules against request {{(pid=80424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:59:49.699933 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48bab98-b2c3-4d93-bcaf-db4510bed483 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39cc268bad604dd08594e3b39c7ecf5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '180b0af5863349e99ae412b07c1e9465', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:59:49.700595 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48bab98-b2c3-4d93-bcaf-db4510bed483 service placement] REQUEST_METHOD: `GET` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:59:49.700882 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48bab98-b2c3-4d93-bcaf-db4510bed483 service placement] SCRIPT_NAME: `/identity` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:59:49.701159 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48bab98-b2c3-4d93-bcaf-db4510bed483 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:59:49.703013 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.fernet_utils [None req-a48bab98-b2c3-4d93-bcaf-db4510bed483 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:59:49.717431 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48bab98-b2c3-4d93-bcaf-db4510bed483 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:59:49.719089 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48bab98-b2c3-4d93-bcaf-db4510bed483 service placement] RBAC: Authorization granted {{(pid=80424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:59:49.731842 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: [pid: 80424|app: 0|req: 1467/2908] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Jun 24 08:59:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:59:59.725646 dv-cinder-847131-2-iscsi devstack@keystone.service[80424]: DEBUG dbcounter [-] [80424] Writing DB stats keystone:SELECT=58 {{(pid=80424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}